[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:22 dist-test-slave-twwt krb5kdc[32704](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:22 dist-test-slave-twwt krb5kdc[32704](info): set up 2 sockets
Aug 08 10:44:22 dist-test-slave-twwt krb5kdc[32704](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 10:44:24.097800 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.436s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:44:24 dist-test-slave-twwt krb5kdc[32704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:44:24.246588 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:43247
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:36461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:43247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:24.369921 32727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:24.370285 32727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:24.370368 32727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:24.374502 32727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:24.374603 32727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:24.374660 32727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:24.374701 32727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:24.374765 32727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:24.380533 32727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:36461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:43247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:43247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:24.382005 32727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:24.383299 32727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:24.393796 32734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:24.393796 32735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:24.393973 32727 server_base.cc:1034] running on GCE node
W20260808 10:44:24.394176 32737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:24.394949 32727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:24.396593 32727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:24.397768 32727 hybrid_clock.cc:648] HybridClock initialized: now 1786185864397757 us; error 53 us; skew 500 ppm
Aug 08 10:44:24 dist-test-slave-twwt krb5kdc[32704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185864, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:24.400939 32727 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:44:24.402024 32727 webserver.cc:460] Webserver started at http://127.31.237.190:37371/ using document root <none> and password file <none>
I20260808 10:44:24.402776 32727 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:24.402882 32727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:24.403280 32727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:24.405628 32727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "4ec1072fe1de43959907b9a3c10d4b5f"
format_stamp: "Formatted at 2026-08-08 10:44:24 on dist-test-slave-twwt"
server_key: "7b60427639e6c99cfb620c4ec0272ebf"
server_key_iv: "faf212e51380b6ee866000e8c8b6c5f4"
server_key_version: "encryptionkey@0"
I20260808 10:44:24.406268 32727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "4ec1072fe1de43959907b9a3c10d4b5f"
format_stamp: "Formatted at 2026-08-08 10:44:24 on dist-test-slave-twwt"
server_key: "7b60427639e6c99cfb620c4ec0272ebf"
server_key_iv: "faf212e51380b6ee866000e8c8b6c5f4"
server_key_version: "encryptionkey@0"
I20260808 10:44:24.410710 32727 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260808 10:44:24.413363 32748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:24.414871 32727 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 10:44:24.415024 32727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "4ec1072fe1de43959907b9a3c10d4b5f"
format_stamp: "Formatted at 2026-08-08 10:44:24 on dist-test-slave-twwt"
server_key: "7b60427639e6c99cfb620c4ec0272ebf"
server_key_iv: "faf212e51380b6ee866000e8c8b6c5f4"
server_key_version: "encryptionkey@0"
I20260808 10:44:24.415151 32727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:24.428529 32727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:24.431728 32727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:24.431952 32727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:24.443434 32727 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:43247
I20260808 10:44:24.443442 367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:43247 every 8 connection(s)
I20260808 10:44:24.444819 32727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:24.448208 368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:24.454762 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 32727
I20260808 10:44:24.454962 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:24.455453 368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f: Bootstrap starting.
I20260808 10:44:24.456759 32694 external_mini_cluster.cc:1428] Setting key 514a685c13cce3b6d1482664ea0d0495
I20260808 10:44:24.459956 368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:24.461678 368 log.cc:824] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:24.464190 368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f: No bootstrap required, opened a new log
I20260808 10:44:24.470388 368 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec1072fe1de43959907b9a3c10d4b5f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43247 } }
I20260808 10:44:24.470657 368 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:24.470731 368 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec1072fe1de43959907b9a3c10d4b5f, State: Initialized, Role: FOLLOWER
I20260808 10:44:24.471266 368 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec1072fe1de43959907b9a3c10d4b5f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43247 } }
I20260808 10:44:24.471407 368 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:24.471503 368 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:24.471612 368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:24.472887 368 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec1072fe1de43959907b9a3c10d4b5f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43247 } }
I20260808 10:44:24.473238 368 leader_election.cc:302] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ec1072fe1de43959907b9a3c10d4b5f; no voters:
I20260808 10:44:24.473598 368 leader_election.cc:288] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:24.473757 376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:24.474052 376 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [term 1 LEADER]: Becoming Leader. State: Replica: 4ec1072fe1de43959907b9a3c10d4b5f, State: Running, Role: LEADER
I20260808 10:44:24.474359 376 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec1072fe1de43959907b9a3c10d4b5f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43247 } }
I20260808 10:44:24.475124 368 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:24.476891 377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ec1072fe1de43959907b9a3c10d4b5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec1072fe1de43959907b9a3c10d4b5f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43247 } } }
I20260808 10:44:24.477044 377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:24.477465 378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ec1072fe1de43959907b9a3c10d4b5f. Latest consensus state: current_term: 1 leader_uuid: "4ec1072fe1de43959907b9a3c10d4b5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec1072fe1de43959907b9a3c10d4b5f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43247 } } }
I20260808 10:44:24.477567 378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:24.477941 380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 10:44:24 dist-test-slave-twwt krb5kdc[32704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:24.483343 380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:24.489101 370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:24.461080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46818 (local address 127.31.237.190:43247)
0808 10:44:24.461780 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:24.461833 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:24.462827 (+ 994us) server_negotiation.cc:407] Connection header received
0808 10:44:24.467696 (+ 4869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:24.467718 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:24.468132 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:24.468488 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:24.471228 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:24.472501 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:24.474118 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:24.474895 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:24.475088 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:24.482161 (+ 7073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:24.482171 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:24.482191 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:24.482237 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:24.484741 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:24.485330 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:24.485343 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:24.485348 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:24.485461 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:24.485996 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:24.486001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:24.486002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:24.486722 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:24.486958 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:24.487712 (+ 754us) server_negotiation.cc:299] Negotiation successful
0808 10:44:24.488031 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":177,"threads_started":1}
I20260808 10:44:24.490454 380 catalog_manager.cc:1347] Generated new cluster ID: 95fd56461208443dae0e1d0e3a49067a
I20260808 10:44:24.490522 380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:24.514557 380 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:24.515537 380 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:24.522953 380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ec1072fe1de43959907b9a3c10d4b5f: Generated new TSK 0
I20260808 10:44:24.523623 380 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:44:24.563565 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:43247
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:36461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:24.707953 409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:24.708246 409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:24.708294 409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:24.712411 409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:24.712471 409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:24.712529 409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:44:24.717890 409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:36461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:43247
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:24.719076 409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:24.720479 409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:24.731415 420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:24.731433 416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:24.731468 417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:24.731746 409 server_base.cc:1034] running on GCE node
I20260808 10:44:24.732126 409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:24.732671 409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:24.733897 409 hybrid_clock.cc:648] HybridClock initialized: now 1786185864733879 us; error 29 us; skew 500 ppm
Aug 08 10:44:24 dist-test-slave-twwt krb5kdc[32704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185864, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:24.736796 409 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:44:24.737648 409 webserver.cc:460] Webserver started at http://127.31.237.129:34869/ using document root <none> and password file <none>
I20260808 10:44:24.738214 409 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:24.738279 409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:24.738523 409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:24.740640 409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "c2683e94656645cfad3635c92c450f2f"
format_stamp: "Formatted at 2026-08-08 10:44:24 on dist-test-slave-twwt"
server_key: "758214b610bbfc5a7aeec221a1dd124d"
server_key_iv: "f764d5891ad8ee9ff751591e9d215d46"
server_key_version: "encryptionkey@0"
I20260808 10:44:24.741271 409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "c2683e94656645cfad3635c92c450f2f"
format_stamp: "Formatted at 2026-08-08 10:44:24 on dist-test-slave-twwt"
server_key: "758214b610bbfc5a7aeec221a1dd124d"
server_key_iv: "f764d5891ad8ee9ff751591e9d215d46"
server_key_version: "encryptionkey@0"
I20260808 10:44:24.745404 409 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:44:24.748075 432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:24.749392 409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:44:24.749557 409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "c2683e94656645cfad3635c92c450f2f"
format_stamp: "Formatted at 2026-08-08 10:44:24 on dist-test-slave-twwt"
server_key: "758214b610bbfc5a7aeec221a1dd124d"
server_key_iv: "f764d5891ad8ee9ff751591e9d215d46"
server_key_version: "encryptionkey@0"
I20260808 10:44:24.749678 409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:24.765938 409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:24.769490 409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:24.769712 409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:24.770390 409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:24.771432 409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:24.771509 409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:24.771608 409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:24.771659 409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:24.785769 409 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:36199
I20260808 10:44:24.785873 603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:36199 every 8 connection(s)
I20260808 10:44:24.786891 409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:44:24.792797 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 409
I20260808 10:44:24.792888 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:44:24.793152 32694 external_mini_cluster.cc:1428] Setting key 5fa83e9c3a91d67050c4e80b8bf73867
Aug 08 10:44:24 dist-test-slave-twwt krb5kdc[32704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185864, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:24.803714 370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:24.789170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:53975 (local address 127.31.237.190:43247)
0808 10:44:24.789332 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:24.789336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:24.790818 (+ 1482us) server_negotiation.cc:407] Connection header received
0808 10:44:24.791761 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:24.791765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:24.791821 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:24.791929 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:24.793485 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:24.794129 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:24.795594 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:24.795837 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:24.795951 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:24.799418 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:24.799424 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:24.799430 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:24.799464 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:24.801175 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:24.801715 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:24.801721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:24.801726 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:24.801830 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:24.802212 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:24.802215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:24.802217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:24.802554 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:24.802733 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:24.803300 (+ 567us) server_negotiation.cc:299] Negotiation successful
0808 10:44:24.803480 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:44:24.804473 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:24.789576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:43247 (local address 127.31.237.129:53975)
0808 10:44:24.790665 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:24.790718 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:24.791553 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:24.792112 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:24.792119 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:24.792578 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:24.793334 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:24.793353 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:24.794258 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:24.794261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:24.795462 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:24.795473 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:24.795993 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:24.795997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:24.796226 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:24.797176 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:24.797210 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:24.799238 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:24.801293 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:24.801298 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:24.801311 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:24.801580 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:24.801934 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:24.801937 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:24.801938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:24.802085 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:24.802717 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:24.802723 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:24.802984 (+ 261us) client_negotiation.cc:769] Sending connection context
0808 10:44:24.803147 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 10:44:24.803417 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"thread_start_us":710,"threads_started":1}
I20260808 10:44:24.805888 605 heartbeater.cc:347] Connected to a master server at 127.31.237.190:43247
I20260808 10:44:24.806165 605 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:44:24.806782 605 heartbeater.cc:511] Master 127.31.237.190:43247 requested a full tablet report, sending...
I20260808 10:44:24.808544 309 ts_manager.cc:194] Registered new tserver with Master: c2683e94656645cfad3635c92c450f2f (127.31.237.129:36199)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
I20260808 10:44:24.809885 309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:53975
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:44:24.827361 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:43247
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:36461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:24.976848 617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:24.977124 617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:24.977170 617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:24.981459 617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:24.981536 617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:24.981630 617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:44:24.987498 617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:36461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:43247
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:24.988819 617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:24.990026 617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:25.000427 633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:25.000383 626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:25.000465 630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:25.000572 617 server_base.cc:1034] running on GCE node
I20260808 10:44:25.001086 617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:25.001788 617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:25.002981 617 hybrid_clock.cc:648] HybridClock initialized: now 1786185865002959 us; error 34 us; skew 500 ppm
Aug 08 10:44:25 dist-test-slave-twwt krb5kdc[32704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185865, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:25.005813 617 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:44:25.006809 617 webserver.cc:460] Webserver started at http://127.31.237.130:32847/ using document root <none> and password file <none>
I20260808 10:44:25.007390 617 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:25.007481 617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:25.007727 617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:25.009809 617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "4df2aff0fd66427eb771babd603a36e9"
format_stamp: "Formatted at 2026-08-08 10:44:25 on dist-test-slave-twwt"
server_key: "a89c1481e2621b0a70a04fa67112a772"
server_key_iv: "bec3e619008a25c4a1f378a8767569f4"
server_key_version: "encryptionkey@0"
I20260808 10:44:25.010449 617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "4df2aff0fd66427eb771babd603a36e9"
format_stamp: "Formatted at 2026-08-08 10:44:25 on dist-test-slave-twwt"
server_key: "a89c1481e2621b0a70a04fa67112a772"
server_key_iv: "bec3e619008a25c4a1f378a8767569f4"
server_key_version: "encryptionkey@0"
I20260808 10:44:25.014483 617 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 10:44:25.017088 643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:25.018381 617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:25.018517 617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "4df2aff0fd66427eb771babd603a36e9"
format_stamp: "Formatted at 2026-08-08 10:44:25 on dist-test-slave-twwt"
server_key: "a89c1481e2621b0a70a04fa67112a772"
server_key_iv: "bec3e619008a25c4a1f378a8767569f4"
server_key_version: "encryptionkey@0"
I20260808 10:44:25.018682 617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:25.036933 617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:25.040359 617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:25.040575 617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:25.041254 617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:25.042137 617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:25.042212 617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:25.042295 617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:25.042348 617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:25.056478 617 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:45937
I20260808 10:44:25.056524 822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:45937 every 8 connection(s)
I20260808 10:44:25.057477 617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:44:25.066010 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 617
I20260808 10:44:25.066182 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:44:25.066433 32694 external_mini_cluster.cc:1428] Setting key 82b63eabc84831205a8a658c5b388d58
Aug 08 10:44:25 dist-test-slave-twwt krb5kdc[32704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185865, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:25.074368 370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:25.059804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:39983 (local address 127.31.237.190:43247)
0808 10:44:25.059954 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:25.059958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:25.060847 (+ 889us) server_negotiation.cc:407] Connection header received
0808 10:44:25.062548 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:25.062552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:25.062645 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:25.062753 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:25.064105 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:25.064661 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:25.065932 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:25.066230 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:25.066355 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:25.070095 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:25.070098 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:25.070100 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:25.070126 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:25.072044 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:25.072590 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:25.072597 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:25.072602 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:25.072698 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:25.073053 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:25.073059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:25.073064 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:25.073324 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:25.073442 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:25.073972 (+ 530us) server_negotiation.cc:299] Negotiation successful
0808 10:44:25.074126 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:44:25.075291 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:25.060180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:43247 (local address 127.31.237.130:39983)
0808 10:44:25.060673 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:25.060726 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:25.062299 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:25.062952 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:25.062960 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:25.063422 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:25.063947 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:25.063962 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:25.064797 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:25.064806 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:25.065740 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:25.065750 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:25.066401 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:25.066405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:25.066675 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:25.067870 (+ 1195us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:25.067892 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:25.069924 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:25.072172 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:25.072178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:25.072192 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:25.072470 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:25.072810 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:25.072812 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:25.072814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:25.072954 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:25.073439 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:25.073445 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:25.073702 (+ 257us) client_negotiation.cc:769] Sending connection context
0808 10:44:25.073876 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 10:44:25.074229 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":118,"threads_started":1}
I20260808 10:44:25.076604 824 heartbeater.cc:347] Connected to a master server at 127.31.237.190:43247
I20260808 10:44:25.076880 824 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:25.077442 824 heartbeater.cc:511] Master 127.31.237.190:43247 requested a full tablet report, sending...
I20260808 10:44:25.078706 309 ts_manager.cc:194] Registered new tserver with Master: 4df2aff0fd66427eb771babd603a36e9 (127.31.237.130:45937)
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:44:25.079385 309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:39983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:44:25.099725 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:43247
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:36461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:25.243358 829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:25.243643 829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:25.243690 829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:25.247987 829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:25.248054 829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:25.248152 829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:44:25.253809 829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:36461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:43247
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:25.254992 829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:25.256091 829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:25.266928 846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:25.266950 837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:25.266908 838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:25.267302 829 server_base.cc:1034] running on GCE node
I20260808 10:44:25.267781 829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:25.268451 829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:25.269667 829 hybrid_clock.cc:648] HybridClock initialized: now 1786185865269642 us; error 43 us; skew 500 ppm
Aug 08 10:44:25 dist-test-slave-twwt krb5kdc[32704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185865, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:25.272699 829 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:44:25.273623 829 webserver.cc:460] Webserver started at http://127.31.237.131:41655/ using document root <none> and password file <none>
I20260808 10:44:25.274206 829 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:25.274297 829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:25.274542 829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:25.276996 829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "18514d871cc6469ea8f2988e2094e84d"
format_stamp: "Formatted at 2026-08-08 10:44:25 on dist-test-slave-twwt"
server_key: "dd05c2596fe45c5dfe1e60ac3179a4e8"
server_key_iv: "dba6462ea35df61a479a03986ff96015"
server_key_version: "encryptionkey@0"
I20260808 10:44:25.277592 829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "18514d871cc6469ea8f2988e2094e84d"
format_stamp: "Formatted at 2026-08-08 10:44:25 on dist-test-slave-twwt"
server_key: "dd05c2596fe45c5dfe1e60ac3179a4e8"
server_key_iv: "dba6462ea35df61a479a03986ff96015"
server_key_version: "encryptionkey@0"
I20260808 10:44:25.281898 829 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 10:44:25.284509 858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:25.285786 829 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:44:25.285935 829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "18514d871cc6469ea8f2988e2094e84d"
format_stamp: "Formatted at 2026-08-08 10:44:25 on dist-test-slave-twwt"
server_key: "dd05c2596fe45c5dfe1e60ac3179a4e8"
server_key_iv: "dba6462ea35df61a479a03986ff96015"
server_key_version: "encryptionkey@0"
I20260808 10:44:25.286053 829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:25.300844 829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:25.304206 829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:25.304469 829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:25.305263 829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:25.306281 829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:25.306367 829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:25.306464 829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:25.306516 829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:25.322330 829 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:38583
I20260808 10:44:25.322386 1030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:38583 every 8 connection(s)
I20260808 10:44:25.323578 829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:44:25.327900 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 829
I20260808 10:44:25.328060 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:44:25.328471 32694 external_mini_cluster.cc:1428] Setting key f72fe87345ce7677d4344a861b538ec2
Aug 08 10:44:25 dist-test-slave-twwt krb5kdc[32704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185865, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:25.340272 370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:25.325839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:40053 (local address 127.31.237.190:43247)
0808 10:44:25.326008 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:25.326013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:25.327232 (+ 1219us) server_negotiation.cc:407] Connection header received
0808 10:44:25.329445 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:25.329448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:25.329528 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:25.329613 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:25.331844 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:25.332391 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:25.333838 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:25.334061 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:25.334141 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:25.336866 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:25.336869 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:25.336871 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:25.336896 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:25.338171 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:25.338752 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:25.338757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:25.338758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:25.338845 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:25.339250 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:25.339254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:25.339255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:25.339464 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:25.339544 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:25.340042 (+ 498us) server_negotiation.cc:299] Negotiation successful
0808 10:44:25.340145 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:44:25.341203 1036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:25.326349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:43247 (local address 127.31.237.131:40053)
0808 10:44:25.327055 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:25.327106 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:25.329166 (+ 2060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:25.329915 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:25.329925 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:25.330763 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:25.331689 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:25.331711 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:25.332534 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:25.332537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:25.333719 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:25.333730 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:25.334198 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:25.334201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:25.334396 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:25.335152 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:25.335171 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:25.336696 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:25.338276 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:25.338282 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:25.338296 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:25.338588 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:25.338964 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:25.338967 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:25.338968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:25.339127 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:25.339557 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:25.339563 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:25.339809 (+ 246us) client_negotiation.cc:769] Sending connection context
0808 10:44:25.339978 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 10:44:25.340260 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":201,"threads_started":1}
I20260808 10:44:25.342469 1031 heartbeater.cc:347] Connected to a master server at 127.31.237.190:43247
I20260808 10:44:25.342798 1031 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:25.343387 1031 heartbeater.cc:511] Master 127.31.237.190:43247 requested a full tablet report, sending...
I20260808 10:44:25.344485 309 ts_manager.cc:194] Registered new tserver with Master: 18514d871cc6469ea8f2988e2094e84d (127.31.237.131:38583)
I20260808 10:44:25.345094 309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:40053
I20260808 10:44:25.355573 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:44:25 dist-test-slave-twwt krb5kdc[32704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185865, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:44:25 dist-test-slave-twwt krb5kdc[32704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185865, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.237.129@KRBTEST.COM
I20260808 10:44:25.385475 1041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:25.375874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56164 (local address 127.31.237.129:36199)
0808 10:44:25.376288 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:25.376294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:25.376319 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:44:25.376375 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:25.376380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:25.376582 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:25.376713 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:25.377342 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:25.378171 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:25.379040 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:25.379468 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:25.379557 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:25.381990 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:25.381998 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:25.382012 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:25.382036 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:25.383787 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:25.384186 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:25.384191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:25.384193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:25.384270 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:25.384508 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:25.384510 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:25.384511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:25.384749 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:25.384916 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:25.385065 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 10:44:25.385238 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":123,"threads_started":1}
W20260808 10:44:25.386485 469 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56164
Aug 08 10:44:25 dist-test-slave-twwt krb5kdc[32704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:44:25 dist-test-slave-twwt krb5kdc[32704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.129@KRBTEST.COM
I20260808 10:44:25.415566 1041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:25.407826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56170 (local address 127.31.237.129:36199)
0808 10:44:25.407984 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:25.407990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:25.408126 (+ 136us) server_negotiation.cc:407] Connection header received
0808 10:44:25.408238 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:25.408241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:25.408283 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:25.408354 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:25.409009 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:25.409501 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:25.410266 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:25.410483 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:25.410561 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:25.412889 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:25.412892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:25.412894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:25.412916 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:25.414135 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:25.414513 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:25.414516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:25.414517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:25.414586 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:25.414883 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:25.414886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:25.414887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:25.415044 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:25.415161 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:25.415341 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 10:44:25.415445 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:44:25.416674 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 409
I20260808 10:44:25.428153 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 617
I20260808 10:44:25.459424 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 829
I20260808 10:44:25.485322 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 32727
2026-08-08T10:44:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2862 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:25 dist-test-slave-twwt krb5kdc[1057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:25 dist-test-slave-twwt krb5kdc[1057](info): set up 2 sockets
Aug 08 10:44:25 dist-test-slave-twwt krb5kdc[1057](info): commencing operation
krb5kdc: starting...
W20260808 10:44:26.754033 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:44:26 dist-test-slave-twwt krb5kdc[1057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:44:26.886437 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:41867
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:41867
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:27.003824 1075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:27.004098 1075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:27.004138 1075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:27.008186 1075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:27.008246 1075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:27.008266 1075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:27.008282 1075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:27.008298 1075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:27.013669 1075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:41867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:41867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:27.014845 1075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:27.015919 1075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:27.025110 1082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:27.025154 1086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:27.025110 1083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:27.025480 1075 server_base.cc:1034] running on GCE node
I20260808 10:44:27.026137 1075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:27.027386 1075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:27.028604 1075 hybrid_clock.cc:648] HybridClock initialized: now 1786185867028593 us; error 32 us; skew 500 ppm
Aug 08 10:44:27 dist-test-slave-twwt krb5kdc[1057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185867, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:27.036691 1075 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:44:27.037724 1075 webserver.cc:460] Webserver started at http://127.31.237.190:34979/ using document root <none> and password file <none>
I20260808 10:44:27.038332 1075 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:27.038426 1075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:27.038746 1075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:27.040963 1075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "a70fc7fb33024ee39ecc44516ab16f94"
format_stamp: "Formatted at 2026-08-08 10:44:27 on dist-test-slave-twwt"
server_key: "016227239935266a0629e9c3b91317c5"
server_key_iv: "4588c482ef8f7b3e9126fa9e1e567e4a"
server_key_version: "encryptionkey@0"
I20260808 10:44:27.041586 1075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "a70fc7fb33024ee39ecc44516ab16f94"
format_stamp: "Formatted at 2026-08-08 10:44:27 on dist-test-slave-twwt"
server_key: "016227239935266a0629e9c3b91317c5"
server_key_iv: "4588c482ef8f7b3e9126fa9e1e567e4a"
server_key_version: "encryptionkey@0"
I20260808 10:44:27.045480 1075 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 10:44:27.048084 1097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:27.049309 1075 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:44:27.049469 1075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "a70fc7fb33024ee39ecc44516ab16f94"
format_stamp: "Formatted at 2026-08-08 10:44:27 on dist-test-slave-twwt"
server_key: "016227239935266a0629e9c3b91317c5"
server_key_iv: "4588c482ef8f7b3e9126fa9e1e567e4a"
server_key_version: "encryptionkey@0"
I20260808 10:44:27.049589 1075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:27.076632 1075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:27.079713 1075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:27.079936 1075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:27.087360 1075 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:41867
I20260808 10:44:27.087389 1173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:41867 every 8 connection(s)
I20260808 10:44:27.088370 1075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:27.091452 1175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:27.093680 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 1075
I20260808 10:44:27.093827 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:27.094094 32694 external_mini_cluster.cc:1428] Setting key 2b480d09b31f0c402c03c3e993393def
I20260808 10:44:27.097625 1175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94: Bootstrap starting.
I20260808 10:44:27.100257 1175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:27.101096 1175 log.cc:824] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:44:27 dist-test-slave-twwt krb5kdc[1057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:27.103627 1175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94: No bootstrap required, opened a new log
I20260808 10:44:27.107227 1175 raft_consensus.cc:348] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70fc7fb33024ee39ecc44516ab16f94" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 41867 } }
I20260808 10:44:27.107486 1175 raft_consensus.cc:374] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:27.107568 1175 raft_consensus.cc:729] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a70fc7fb33024ee39ecc44516ab16f94, State: Initialized, Role: FOLLOWER
I20260808 10:44:27.108127 1175 consensus_queue.cc:260] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70fc7fb33024ee39ecc44516ab16f94" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 41867 } }
I20260808 10:44:27.108323 1175 raft_consensus.cc:388] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:27.108397 1175 raft_consensus.cc:482] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:27.108510 1175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:27.109272 1180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:27.095584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60900 (local address 127.31.237.190:41867)
0808 10:44:27.096000 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:27.096022 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:27.096059 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:44:27.096704 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:27.096721 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:27.097066 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:27.097348 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:27.098271 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.099322 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.100487 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.100969 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.101099 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:27.104244 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:27.104250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:27.104264 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:27.104292 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:27.106086 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:27.106575 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:27.106580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:27.106585 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:27.106759 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:27.107058 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:27.107062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:27.107064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:27.107483 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:27.107741 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:27.108035 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 10:44:27.108403 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":136,"threads_started":1}
I20260808 10:44:27.109503 1175 raft_consensus.cc:504] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70fc7fb33024ee39ecc44516ab16f94" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 41867 } }
I20260808 10:44:27.109895 1175 leader_election.cc:302] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a70fc7fb33024ee39ecc44516ab16f94; no voters:
I20260808 10:44:27.110301 1175 leader_election.cc:288] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:27.110467 1182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:27.110705 1182 raft_consensus.cc:686] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [term 1 LEADER]: Becoming Leader. State: Replica: a70fc7fb33024ee39ecc44516ab16f94, State: Running, Role: LEADER
I20260808 10:44:27.111099 1182 consensus_queue.cc:237] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70fc7fb33024ee39ecc44516ab16f94" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 41867 } }
I20260808 10:44:27.111756 1175 sys_catalog.cc:564] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:27.112385 1182 sys_catalog.cc:455] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a70fc7fb33024ee39ecc44516ab16f94. Latest consensus state: current_term: 1 leader_uuid: "a70fc7fb33024ee39ecc44516ab16f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70fc7fb33024ee39ecc44516ab16f94" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 41867 } } }
I20260808 10:44:27.112526 1182 sys_catalog.cc:458] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:27.112854 1183 sys_catalog.cc:455] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a70fc7fb33024ee39ecc44516ab16f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70fc7fb33024ee39ecc44516ab16f94" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 41867 } } }
I20260808 10:44:27.112980 1183 sys_catalog.cc:458] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94 [sys.catalog]: This master's current role is: LEADER
W20260808 10:44:27.117096 1209 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:27.117188 1209 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:27.117260 1189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:27.117985 1189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:27.123723 1189 catalog_manager.cc:1347] Generated new cluster ID: 34c894222feb4f7a9b6ff87c85f51dcc
I20260808 10:44:27.123805 1189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:27.135259 1189 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:27.136639 1189 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:27.142103 1189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a70fc7fb33024ee39ecc44516ab16f94: Generated new TSK 0
I20260808 10:44:27.143031 1189 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:44:27.187019 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:41867
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:27.327989 1219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:27.328266 1219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:27.328310 1219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:27.332365 1219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:27.332424 1219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:27.332495 1219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:44:27.337977 1219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:41867
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:27.339069 1219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:27.340058 1219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:27.350307 1227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:27.350307 1228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:27.350338 1232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:27.350790 1219 server_base.cc:1034] running on GCE node
I20260808 10:44:27.351210 1219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:27.351816 1219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:27.352996 1219 hybrid_clock.cc:648] HybridClock initialized: now 1786185867352971 us; error 28 us; skew 500 ppm
Aug 08 10:44:27 dist-test-slave-twwt krb5kdc[1057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185867, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:27.356066 1219 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:44:27.357024 1219 webserver.cc:460] Webserver started at http://127.31.237.129:43113/ using document root <none> and password file <none>
I20260808 10:44:27.357609 1219 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:27.357702 1219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:27.357954 1219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:27.360142 1219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "ab81b56721004d4c986c0aef9b384116"
format_stamp: "Formatted at 2026-08-08 10:44:27 on dist-test-slave-twwt"
server_key: "98b4f0dc8321f5e7520f3043d42b6c75"
server_key_iv: "75021958449fb6116b05f3affbbcfe02"
server_key_version: "encryptionkey@0"
I20260808 10:44:27.360723 1219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "ab81b56721004d4c986c0aef9b384116"
format_stamp: "Formatted at 2026-08-08 10:44:27 on dist-test-slave-twwt"
server_key: "98b4f0dc8321f5e7520f3043d42b6c75"
server_key_iv: "75021958449fb6116b05f3affbbcfe02"
server_key_version: "encryptionkey@0"
I20260808 10:44:27.364483 1219 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:44:27.366914 1243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:27.368101 1219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:44:27.368245 1219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "ab81b56721004d4c986c0aef9b384116"
format_stamp: "Formatted at 2026-08-08 10:44:27 on dist-test-slave-twwt"
server_key: "98b4f0dc8321f5e7520f3043d42b6c75"
server_key_iv: "75021958449fb6116b05f3affbbcfe02"
server_key_version: "encryptionkey@0"
I20260808 10:44:27.368362 1219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:27.405318 1219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:27.408480 1219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:27.408704 1219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:27.409354 1219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:27.410262 1219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:27.410338 1219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:27.410415 1219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:27.410472 1219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:27.420045 1219 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:45001
I20260808 10:44:27.420070 1418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:45001 every 8 connection(s)
I20260808 10:44:27.420969 1219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:44:27.425818 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 1219
I20260808 10:44:27.425909 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:44:27.426119 32694 external_mini_cluster.cc:1428] Setting key b29edaf6a90bdfcd78251a69fe01465f
Aug 08 10:44:27 dist-test-slave-twwt krb5kdc[1057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185867, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:27.436100 1180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:27.422944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:56323 (local address 127.31.237.190:41867)
0808 10:44:27.423084 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:27.423088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:27.423978 (+ 890us) server_negotiation.cc:407] Connection header received
0808 10:44:27.425426 (+ 1448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:27.425429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:27.425486 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:27.425596 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:27.427676 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.428441 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.429616 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.429861 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.429991 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:27.432508 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:27.432518 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:27.432521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:27.432543 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:27.433995 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:27.434505 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:27.434510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:27.434512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:27.434595 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:27.434997 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:27.435002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:27.435004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:27.435190 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:27.435299 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:27.435795 (+ 496us) server_negotiation.cc:299] Negotiation successful
0808 10:44:27.435924 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:44:27.436954 1422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:27.423364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:41867 (local address 127.31.237.129:56323)
0808 10:44:27.423813 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:27.423858 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:27.425206 (+ 1348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:27.426419 (+ 1213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:27.426427 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:27.426914 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:27.427503 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:27.427526 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.428572 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.428575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:27.429489 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:27.429500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.430031 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.430035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:27.430216 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:27.430877 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:27.430894 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:27.432340 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:27.434115 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:27.434121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:27.434135 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:27.434400 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:27.434739 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:27.434742 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:27.434744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:27.434888 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:27.435300 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:27.435307 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:27.435553 (+ 246us) client_negotiation.cc:769] Sending connection context
0808 10:44:27.435739 (+ 186us) client_negotiation.cc:240] Negotiation successful
0808 10:44:27.436069 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":115,"threads_started":1}
I20260808 10:44:27.438241 1419 heartbeater.cc:347] Connected to a master server at 127.31.237.190:41867
I20260808 10:44:27.438509 1419 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:27.439090 1419 heartbeater.cc:511] Master 127.31.237.190:41867 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
I20260808 10:44:27.440804 1120 ts_manager.cc:194] Registered new tserver with Master: ab81b56721004d4c986c0aef9b384116 (127.31.237.129:45001)
I20260808 10:44:27.441977 1120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:56323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:44:27.457262 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:41867
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:27.577261 1428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:27.577606 1428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:27.577695 1428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:27.581879 1428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:27.581998 1428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:27.582108 1428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:44:27.587734 1428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:41867
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:27.588946 1428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:27.590386 1428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:27.601150 1437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:27.601138 1440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:27.601142 1438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:27.601198 1428 server_base.cc:1034] running on GCE node
I20260808 10:44:27.601851 1428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:27.602593 1428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:27.603844 1428 hybrid_clock.cc:648] HybridClock initialized: now 1786185867603825 us; error 32 us; skew 500 ppm
Aug 08 10:44:27 dist-test-slave-twwt krb5kdc[1057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185867, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:27.606927 1428 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:44:27.607910 1428 webserver.cc:460] Webserver started at http://127.31.237.130:39967/ using document root <none> and password file <none>
I20260808 10:44:27.608549 1428 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:27.608650 1428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:27.608907 1428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:27.611222 1428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "60f309e3628a469eb2dcbabae779adfc"
format_stamp: "Formatted at 2026-08-08 10:44:27 on dist-test-slave-twwt"
server_key: "7c75c8208f9090ce2b01177cb225a680"
server_key_iv: "7e9a77b122f149d8555389726be005ee"
server_key_version: "encryptionkey@0"
I20260808 10:44:27.611891 1428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "60f309e3628a469eb2dcbabae779adfc"
format_stamp: "Formatted at 2026-08-08 10:44:27 on dist-test-slave-twwt"
server_key: "7c75c8208f9090ce2b01177cb225a680"
server_key_iv: "7e9a77b122f149d8555389726be005ee"
server_key_version: "encryptionkey@0"
I20260808 10:44:27.616351 1428 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 10:44:27.619052 1450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:27.620398 1428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:44:27.620563 1428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "60f309e3628a469eb2dcbabae779adfc"
format_stamp: "Formatted at 2026-08-08 10:44:27 on dist-test-slave-twwt"
server_key: "7c75c8208f9090ce2b01177cb225a680"
server_key_iv: "7e9a77b122f149d8555389726be005ee"
server_key_version: "encryptionkey@0"
I20260808 10:44:27.620699 1428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:27.644703 1428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:27.648257 1428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:27.648486 1428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:27.649132 1428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:27.650017 1428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:27.650064 1428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:27.650135 1428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:27.650171 1428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:27.659323 1428 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:38905
I20260808 10:44:27.659394 1618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:38905 every 8 connection(s)
I20260808 10:44:27.660283 1428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:44:27.665282 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 1428
I20260808 10:44:27.665383 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:44:27.665648 32694 external_mini_cluster.cc:1428] Setting key 565fe20aa5babae4012b3d56980f8caa
Aug 08 10:44:27 dist-test-slave-twwt krb5kdc[1057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185867, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:27.676412 1180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:27.662293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:44237 (local address 127.31.237.190:41867)
0808 10:44:27.662486 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:27.662492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:27.663477 (+ 985us) server_negotiation.cc:407] Connection header received
0808 10:44:27.664858 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:27.664861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:27.664913 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:27.664994 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:27.666524 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.667306 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.669140 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.669347 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.669453 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:27.672380 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:27.672391 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:27.672394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:27.672420 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:27.673988 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:27.674550 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:27.674554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:27.674556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:27.674702 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:27.675082 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:27.675087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:27.675088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:27.675295 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:27.675412 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:27.676091 (+ 679us) server_negotiation.cc:299] Negotiation successful
0808 10:44:27.676226 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:44:27.677384 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:27.662802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:41867 (local address 127.31.237.130:44237)
0808 10:44:27.663300 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:27.663346 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:27.664633 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:27.665200 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:27.665208 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:27.665709 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:27.666342 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:27.666365 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.667448 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.667451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:27.669001 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:27.669017 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.669517 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.669524 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:27.669765 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:27.670417 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:27.670434 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:27.672195 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:27.674128 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:27.674134 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:27.674153 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:27.674435 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:27.674807 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:27.674810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:27.674812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:27.674978 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:27.675412 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:27.675421 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:27.675812 (+ 391us) client_negotiation.cc:769] Sending connection context
0808 10:44:27.676044 (+ 232us) client_negotiation.cc:240] Negotiation successful
0808 10:44:27.676410 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":124,"threads_started":1}
I20260808 10:44:27.678787 1620 heartbeater.cc:347] Connected to a master server at 127.31.237.190:41867
I20260808 10:44:27.679057 1620 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:27.679639 1620 heartbeater.cc:511] Master 127.31.237.190:41867 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
I20260808 10:44:27.680871 1120 ts_manager.cc:194] Registered new tserver with Master: 60f309e3628a469eb2dcbabae779adfc (127.31.237.130:38905)
I20260808 10:44:27.681740 1120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:44237
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:44:27.697885 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:41867
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:27.817936 1628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:27.818274 1628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:27.818359 1628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:27.822706 1628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:27.822794 1628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:27.822885 1628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:44:27.828447 1628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:41867
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:27.829591 1628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:27.830709 1628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:27.840767 1637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:27.840899 1641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:27.840997 1628 server_base.cc:1034] running on GCE node
W20260808 10:44:27.841012 1635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:27.841637 1628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:27.842412 1628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:27.843627 1628 hybrid_clock.cc:648] HybridClock initialized: now 1786185867843605 us; error 40 us; skew 500 ppm
Aug 08 10:44:27 dist-test-slave-twwt krb5kdc[1057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185867, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:27.846798 1628 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:44:27.847756 1628 webserver.cc:460] Webserver started at http://127.31.237.131:35955/ using document root <none> and password file <none>
I20260808 10:44:27.848368 1628 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:27.848469 1628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:27.848721 1628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:27.850908 1628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "63eeeb64a5d542de87b1e2511c37bf96"
format_stamp: "Formatted at 2026-08-08 10:44:27 on dist-test-slave-twwt"
server_key: "be02e3b419bfafefd181c23e41e99fed"
server_key_iv: "0ac4fa0c54d04d8a9cc294d9e957b074"
server_key_version: "encryptionkey@0"
I20260808 10:44:27.851502 1628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "63eeeb64a5d542de87b1e2511c37bf96"
format_stamp: "Formatted at 2026-08-08 10:44:27 on dist-test-slave-twwt"
server_key: "be02e3b419bfafefd181c23e41e99fed"
server_key_iv: "0ac4fa0c54d04d8a9cc294d9e957b074"
server_key_version: "encryptionkey@0"
I20260808 10:44:27.855839 1628 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:44:27.858415 1650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:27.859910 1628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:44:27.860121 1628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "63eeeb64a5d542de87b1e2511c37bf96"
format_stamp: "Formatted at 2026-08-08 10:44:27 on dist-test-slave-twwt"
server_key: "be02e3b419bfafefd181c23e41e99fed"
server_key_iv: "0ac4fa0c54d04d8a9cc294d9e957b074"
server_key_version: "encryptionkey@0"
I20260808 10:44:27.860261 1628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:27.882684 1628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:27.886214 1628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:27.886443 1628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:27.887154 1628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:27.888067 1628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:27.888141 1628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:27.888214 1628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:27.888258 1628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:27.897430 1628 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:32797
I20260808 10:44:27.897454 1824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:32797 every 8 connection(s)
I20260808 10:44:27.898367 1628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:44:27.905619 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 1628
I20260808 10:44:27.905771 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:44:27.906066 32694 external_mini_cluster.cc:1428] Setting key 9428c99e339585c5fbabe8146bc3b5c7
Aug 08 10:44:27 dist-test-slave-twwt krb5kdc[1057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185867, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:27.912513 1180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:27.900350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:50537 (local address 127.31.237.190:41867)
0808 10:44:27.900500 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:27.900504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:27.901368 (+ 864us) server_negotiation.cc:407] Connection header received
0808 10:44:27.902768 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:27.902771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:27.902822 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:27.902897 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:27.904073 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.904605 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.905712 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.906045 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.906160 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:27.908857 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:27.908861 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:27.908863 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:27.908885 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:27.910341 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:27.910879 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:27.910881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:27.910882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:27.910957 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:27.911329 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:27.911335 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:27.911336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:27.911536 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:27.911650 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:27.912148 (+ 498us) server_negotiation.cc:299] Negotiation successful
0808 10:44:27.912268 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:44:27.913334 1828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:27.900683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:41867 (local address 127.31.237.131:50537)
0808 10:44:27.901144 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:27.901188 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:27.902507 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:27.903071 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:27.903078 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:27.903430 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:27.903936 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:27.903950 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.904716 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.904718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:27.905584 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:27.905593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.906224 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.906227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:27.906374 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:27.907134 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:27.907151 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:27.908707 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:27.910441 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:27.910445 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:27.910463 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:27.910787 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:27.911036 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:27.911038 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:27.911039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:27.911172 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:27.911651 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:27.911659 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:27.911900 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 10:44:27.912087 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 10:44:27.912425 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":122,"threads_started":1}
I20260808 10:44:27.914458 1825 heartbeater.cc:347] Connected to a master server at 127.31.237.190:41867
I20260808 10:44:27.914726 1825 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:27.915190 1825 heartbeater.cc:511] Master 127.31.237.190:41867 requested a full tablet report, sending...
I20260808 10:44:27.916150 1120 ts_manager.cc:194] Registered new tserver with Master: 63eeeb64a5d542de87b1e2511c37bf96 (127.31.237.131:32797)
I20260808 10:44:27.916723 1120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:50537
I20260808 10:44:27.919971 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:44:27.931594 1180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:27.923490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60902 (local address 127.31.237.190:41867)
0808 10:44:27.923636 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:27.923641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:27.923815 (+ 174us) server_negotiation.cc:407] Connection header received
0808 10:44:27.923945 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:27.923948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:27.923989 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:27.924054 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:27.924673 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.925217 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.926043 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.926371 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.926505 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:27.928446 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:27.928452 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:27.928455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:27.928483 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:27.930143 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:27.930544 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:27.930547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:27.930548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:27.930665 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:27.930892 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:27.930894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:27.930895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:27.931100 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:27.931200 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:27.931387 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 10:44:27.931476 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:44:27.936443 1120 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:44:27.939574 1120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:44:27.955389 1844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:27.948903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:38905 (local address 127.0.0.1:51314)
0808 10:44:27.949525 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:27.949542 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:27.949635 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:27.950871 (+ 1236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:27.950874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:27.950893 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:27.950984 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:27.950990 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.952599 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.952603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:27.953714 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:27.953724 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.954843 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.954847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:27.954969 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:27.955008 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 10:44:27.955100 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 10:44:27.955236 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260808 10:44:27.955960 1841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:27.948484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:45001 (local address 127.0.0.1:51844)
0808 10:44:27.949121 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:27.949168 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:27.949300 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:27.950117 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:27.950125 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:27.950152 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:27.950297 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:27.950307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.951813 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.951817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:27.953586 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:27.953595 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.954843 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.954847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:27.955663 (+ 816us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:27.955687 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:44:27.955759 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 10:44:27.955828 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":708,"thread_start_us":107,"threads_started":1}
I20260808 10:44:27.956434 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:27.948675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51844 (local address 127.31.237.129:45001)
0808 10:44:27.948997 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:27.949007 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:27.949466 (+ 459us) server_negotiation.cc:407] Connection header received
0808 10:44:27.949554 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:27.949561 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:27.949776 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:27.949929 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:27.950440 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.951663 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.953736 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.954701 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.954907 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:27.955060 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:27.956182 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0808 10:44:27.956288 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260808 10:44:27.956610 1847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:27.949447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51314 (local address 127.31.237.130:38905)
0808 10:44:27.950335 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:27.950344 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:27.950385 (+ 41us) server_negotiation.cc:407] Connection header received
0808 10:44:27.950473 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:27.950479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:27.950711 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:27.951133 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:27.951309 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.952441 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.953878 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.954694 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.956174 (+ 1480us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:27.956295 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:27.956342 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 10:44:27.956423 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":114,"threads_started":1}
I20260808 10:44:27.957051 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:27.949256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:32797 (local address 127.0.0.1:48620)
0808 10:44:27.950898 (+ 1642us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:27.950920 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:27.951133 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:27.951997 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:27.952000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:27.952018 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:27.952165 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:27.952172 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.954172 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.954176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:27.955548 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:27.955559 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.956710 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.956713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:27.956794 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:27.956813 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:44:27.956875 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 10:44:27.956932 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1509,"thread_start_us":106,"threads_started":1}
I20260808 10:44:27.959823 1516 tablet_service.cc:1467] Processing CreateTablet for tablet 8cf4b22266b94d7da2db1f6ca6229079 (DEFAULT_TABLE table=test-table [id=3b81d8a859da4748a341207f1c3ba32a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:27.960989 1516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cf4b22266b94d7da2db1f6ca6229079. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:27.963447 1848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:27.949861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48620 (local address 127.31.237.131:32797)
0808 10:44:27.950638 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:27.950652 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:27.951024 (+ 372us) server_negotiation.cc:407] Connection header received
0808 10:44:27.951371 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:27.951380 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:27.951776 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:27.952278 (+ 502us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:27.952585 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.953998 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.955678 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:27.956426 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:27.956547 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:27.956650 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:27.957195 (+ 545us) server_negotiation.cc:299] Negotiation successful
0808 10:44:27.957281 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":159,"threads_started":1}
I20260808 10:44:27.967151 1722 tablet_service.cc:1467] Processing CreateTablet for tablet 8cf4b22266b94d7da2db1f6ca6229079 (DEFAULT_TABLE table=test-table [id=3b81d8a859da4748a341207f1c3ba32a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:27.968389 1722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cf4b22266b94d7da2db1f6ca6229079. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:27.968575 1319 tablet_service.cc:1467] Processing CreateTablet for tablet 8cf4b22266b94d7da2db1f6ca6229079 (DEFAULT_TABLE table=test-table [id=3b81d8a859da4748a341207f1c3ba32a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:27.969611 1319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cf4b22266b94d7da2db1f6ca6229079. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:27.970804 1850 tablet_bootstrap.cc:492] T 8cf4b22266b94d7da2db1f6ca6229079 P 60f309e3628a469eb2dcbabae779adfc: Bootstrap starting.
I20260808 10:44:27.973481 1850 tablet_bootstrap.cc:654] T 8cf4b22266b94d7da2db1f6ca6229079 P 60f309e3628a469eb2dcbabae779adfc: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:27.974534 1850 log.cc:824] T 8cf4b22266b94d7da2db1f6ca6229079 P 60f309e3628a469eb2dcbabae779adfc: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:27.976676 1850 tablet_bootstrap.cc:492] T 8cf4b22266b94d7da2db1f6ca6229079 P 60f309e3628a469eb2dcbabae779adfc: No bootstrap required, opened a new log
I20260808 10:44:27.976888 1850 ts_tablet_manager.cc:1397] T 8cf4b22266b94d7da2db1f6ca6229079 P 60f309e3628a469eb2dcbabae779adfc: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260808 10:44:27.976889 1853 tablet_bootstrap.cc:492] T 8cf4b22266b94d7da2db1f6ca6229079 P 63eeeb64a5d542de87b1e2511c37bf96: Bootstrap starting.
I20260808 10:44:27.977746 1855 tablet_bootstrap.cc:492] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116: Bootstrap starting.
I20260808 10:44:27.980084 1853 tablet_bootstrap.cc:654] T 8cf4b22266b94d7da2db1f6ca6229079 P 63eeeb64a5d542de87b1e2511c37bf96: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:27.980450 1855 tablet_bootstrap.cc:654] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:27.981462 1855 log.cc:824] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:27.981678 1853 log.cc:824] T 8cf4b22266b94d7da2db1f6ca6229079 P 63eeeb64a5d542de87b1e2511c37bf96: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:27.981730 1850 raft_consensus.cc:348] T 8cf4b22266b94d7da2db1f6ca6229079 P 60f309e3628a469eb2dcbabae779adfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab81b56721004d4c986c0aef9b384116" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45001 } } peers { permanent_uuid: "63eeeb64a5d542de87b1e2511c37bf96" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 32797 } } peers { permanent_uuid: "60f309e3628a469eb2dcbabae779adfc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38905 } }
I20260808 10:44:27.982039 1850 raft_consensus.cc:374] T 8cf4b22266b94d7da2db1f6ca6229079 P 60f309e3628a469eb2dcbabae779adfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:27.982120 1850 raft_consensus.cc:729] T 8cf4b22266b94d7da2db1f6ca6229079 P 60f309e3628a469eb2dcbabae779adfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60f309e3628a469eb2dcbabae779adfc, State: Initialized, Role: FOLLOWER
I20260808 10:44:27.982709 1850 consensus_queue.cc:260] T 8cf4b22266b94d7da2db1f6ca6229079 P 60f309e3628a469eb2dcbabae779adfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab81b56721004d4c986c0aef9b384116" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45001 } } peers { permanent_uuid: "63eeeb64a5d542de87b1e2511c37bf96" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 32797 } } peers { permanent_uuid: "60f309e3628a469eb2dcbabae779adfc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38905 } }
I20260808 10:44:27.983590 1620 heartbeater.cc:503] Master 127.31.237.190:41867 was elected leader, sending a full tablet report...
I20260808 10:44:27.983762 1850 ts_tablet_manager.cc:1428] T 8cf4b22266b94d7da2db1f6ca6229079 P 60f309e3628a469eb2dcbabae779adfc: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260808 10:44:27.984019 1855 tablet_bootstrap.cc:492] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116: No bootstrap required, opened a new log
I20260808 10:44:27.984107 1853 tablet_bootstrap.cc:492] T 8cf4b22266b94d7da2db1f6ca6229079 P 63eeeb64a5d542de87b1e2511c37bf96: No bootstrap required, opened a new log
I20260808 10:44:27.984174 1855 ts_tablet_manager.cc:1397] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 10:44:27.984338 1853 ts_tablet_manager.cc:1397] T 8cf4b22266b94d7da2db1f6ca6229079 P 63eeeb64a5d542de87b1e2511c37bf96: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260808 10:44:27.988905 1853 raft_consensus.cc:348] T 8cf4b22266b94d7da2db1f6ca6229079 P 63eeeb64a5d542de87b1e2511c37bf96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab81b56721004d4c986c0aef9b384116" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45001 } } peers { permanent_uuid: "63eeeb64a5d542de87b1e2511c37bf96" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 32797 } } peers { permanent_uuid: "60f309e3628a469eb2dcbabae779adfc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38905 } }
I20260808 10:44:27.989224 1853 raft_consensus.cc:374] T 8cf4b22266b94d7da2db1f6ca6229079 P 63eeeb64a5d542de87b1e2511c37bf96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:27.989293 1853 raft_consensus.cc:729] T 8cf4b22266b94d7da2db1f6ca6229079 P 63eeeb64a5d542de87b1e2511c37bf96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63eeeb64a5d542de87b1e2511c37bf96, State: Initialized, Role: FOLLOWER
I20260808 10:44:27.989223 1855 raft_consensus.cc:348] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab81b56721004d4c986c0aef9b384116" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45001 } } peers { permanent_uuid: "63eeeb64a5d542de87b1e2511c37bf96" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 32797 } } peers { permanent_uuid: "60f309e3628a469eb2dcbabae779adfc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38905 } }
I20260808 10:44:27.989514 1855 raft_consensus.cc:374] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:27.989578 1855 raft_consensus.cc:729] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab81b56721004d4c986c0aef9b384116, State: Initialized, Role: FOLLOWER
I20260808 10:44:27.989815 1853 consensus_queue.cc:260] T 8cf4b22266b94d7da2db1f6ca6229079 P 63eeeb64a5d542de87b1e2511c37bf96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab81b56721004d4c986c0aef9b384116" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45001 } } peers { permanent_uuid: "63eeeb64a5d542de87b1e2511c37bf96" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 32797 } } peers { permanent_uuid: "60f309e3628a469eb2dcbabae779adfc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38905 } }
I20260808 10:44:27.990010 1855 consensus_queue.cc:260] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab81b56721004d4c986c0aef9b384116" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45001 } } peers { permanent_uuid: "63eeeb64a5d542de87b1e2511c37bf96" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 32797 } } peers { permanent_uuid: "60f309e3628a469eb2dcbabae779adfc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38905 } }
I20260808 10:44:27.990546 1825 heartbeater.cc:503] Master 127.31.237.190:41867 was elected leader, sending a full tablet report...
I20260808 10:44:27.990707 1419 heartbeater.cc:503] Master 127.31.237.190:41867 was elected leader, sending a full tablet report...
I20260808 10:44:27.990906 1853 ts_tablet_manager.cc:1428] T 8cf4b22266b94d7da2db1f6ca6229079 P 63eeeb64a5d542de87b1e2511c37bf96: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260808 10:44:27.992115 1855 ts_tablet_manager.cc:1428] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260808 10:44:28.038398 1863 raft_consensus.cc:482] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:44:28.038776 1863 raft_consensus.cc:504] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab81b56721004d4c986c0aef9b384116" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45001 } } peers { permanent_uuid: "63eeeb64a5d542de87b1e2511c37bf96" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 32797 } } peers { permanent_uuid: "60f309e3628a469eb2dcbabae779adfc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38905 } }
I20260808 10:44:28.040082 1863 leader_election.cc:288] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63eeeb64a5d542de87b1e2511c37bf96 (127.31.237.131:32797), 60f309e3628a469eb2dcbabae779adfc (127.31.237.130:38905)
I20260808 10:44:28.044144 1848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:28.040478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:44211 (local address 127.31.237.131:32797)
0808 10:44:28.040779 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:28.040786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:28.040859 (+ 73us) server_negotiation.cc:407] Connection header received
0808 10:44:28.041030 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:28.041034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:28.041085 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:28.041173 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:28.041487 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:28.041983 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:28.043157 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:28.043576 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:28.043710 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:28.043769 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:28.043945 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 10:44:28.044019 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260808 10:44:28.044173 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:28.040345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:32797 (local address 127.31.237.129:44211)
0808 10:44:28.040752 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:28.040769 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:28.040907 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:28.041189 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:28.041192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:28.041209 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:28.041362 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:28.041368 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:28.042104 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:28.042107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:28.043020 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:28.043031 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:28.043703 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:28.043707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:28.043804 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:28.043835 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 10:44:28.043926 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 10:44:28.044004 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":2,"thread_start_us":172,"threads_started":1}
I20260808 10:44:28.044535 1868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:28.040422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:38905 (local address 127.31.237.129:43727)
0808 10:44:28.040877 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:28.040890 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:28.041029 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:28.041274 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:28.041276 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:28.041293 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:28.041444 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:28.041450 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:28.042166 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:28.042170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:28.043166 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:28.043176 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:28.043939 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:28.043942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:28.044138 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:28.044157 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:44:28.044252 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 10:44:28.044342 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":24,"thread_start_us":169,"threads_started":1}
I20260808 10:44:28.044785 1847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:28.040483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:43727 (local address 127.31.237.130:38905)
0808 10:44:28.040659 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:28.040667 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:28.040992 (+ 325us) server_negotiation.cc:407] Connection header received
0808 10:44:28.041120 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:28.041123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:28.041171 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:28.041261 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:28.041563 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:28.042051 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:28.043294 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:28.043802 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:28.043938 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:28.043990 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:28.044592 (+ 602us) server_negotiation.cc:299] Negotiation successful
0808 10:44:28.044674 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 10:44:28.044836 1754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cf4b22266b94d7da2db1f6ca6229079" candidate_uuid: "ab81b56721004d4c986c0aef9b384116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63eeeb64a5d542de87b1e2511c37bf96" is_pre_election: true
I20260808 10:44:28.045187 1754 raft_consensus.cc:2454] T 8cf4b22266b94d7da2db1f6ca6229079 P 63eeeb64a5d542de87b1e2511c37bf96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab81b56721004d4c986c0aef9b384116 in term 0.
I20260808 10:44:28.045491 1555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cf4b22266b94d7da2db1f6ca6229079" candidate_uuid: "ab81b56721004d4c986c0aef9b384116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60f309e3628a469eb2dcbabae779adfc" is_pre_election: true
I20260808 10:44:28.045739 1247 leader_election.cc:302] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63eeeb64a5d542de87b1e2511c37bf96, ab81b56721004d4c986c0aef9b384116; no voters:
I20260808 10:44:28.045774 1555 raft_consensus.cc:2454] T 8cf4b22266b94d7da2db1f6ca6229079 P 60f309e3628a469eb2dcbabae779adfc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab81b56721004d4c986c0aef9b384116 in term 0.
I20260808 10:44:28.046031 1863 raft_consensus.cc:2781] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:44:28.046135 1863 raft_consensus.cc:482] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:44:28.046213 1863 raft_consensus.cc:3037] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:28.047669 1863 raft_consensus.cc:504] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab81b56721004d4c986c0aef9b384116" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45001 } } peers { permanent_uuid: "63eeeb64a5d542de87b1e2511c37bf96" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 32797 } } peers { permanent_uuid: "60f309e3628a469eb2dcbabae779adfc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38905 } }
I20260808 10:44:28.048004 1863 leader_election.cc:288] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [CANDIDATE]: Term 1 election: Requested vote from peers 63eeeb64a5d542de87b1e2511c37bf96 (127.31.237.131:32797), 60f309e3628a469eb2dcbabae779adfc (127.31.237.130:38905)
I20260808 10:44:28.048377 1754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cf4b22266b94d7da2db1f6ca6229079" candidate_uuid: "ab81b56721004d4c986c0aef9b384116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63eeeb64a5d542de87b1e2511c37bf96"
I20260808 10:44:28.048458 1555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cf4b22266b94d7da2db1f6ca6229079" candidate_uuid: "ab81b56721004d4c986c0aef9b384116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60f309e3628a469eb2dcbabae779adfc"
I20260808 10:44:28.048529 1754 raft_consensus.cc:3037] T 8cf4b22266b94d7da2db1f6ca6229079 P 63eeeb64a5d542de87b1e2511c37bf96 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:28.048594 1555 raft_consensus.cc:3037] T 8cf4b22266b94d7da2db1f6ca6229079 P 60f309e3628a469eb2dcbabae779adfc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:28.049758 1555 raft_consensus.cc:2454] T 8cf4b22266b94d7da2db1f6ca6229079 P 60f309e3628a469eb2dcbabae779adfc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab81b56721004d4c986c0aef9b384116 in term 1.
I20260808 10:44:28.049966 1754 raft_consensus.cc:2454] T 8cf4b22266b94d7da2db1f6ca6229079 P 63eeeb64a5d542de87b1e2511c37bf96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab81b56721004d4c986c0aef9b384116 in term 1.
I20260808 10:44:28.050104 1246 leader_election.cc:302] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60f309e3628a469eb2dcbabae779adfc, ab81b56721004d4c986c0aef9b384116; no voters:
I20260808 10:44:28.050299 1863 raft_consensus.cc:2781] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:28.050520 1863 raft_consensus.cc:686] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [term 1 LEADER]: Becoming Leader. State: Replica: ab81b56721004d4c986c0aef9b384116, State: Running, Role: LEADER
I20260808 10:44:28.050873 1863 consensus_queue.cc:237] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab81b56721004d4c986c0aef9b384116" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45001 } } peers { permanent_uuid: "63eeeb64a5d542de87b1e2511c37bf96" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 32797 } } peers { permanent_uuid: "60f309e3628a469eb2dcbabae779adfc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38905 } }
I20260808 10:44:28.054445 1120 catalog_manager.cc:5495] T 8cf4b22266b94d7da2db1f6ca6229079 P ab81b56721004d4c986c0aef9b384116 reported cstate change: term changed from 0 to 1, leader changed from <none> to ab81b56721004d4c986c0aef9b384116 (127.31.237.129). New cstate: current_term: 1 leader_uuid: "ab81b56721004d4c986c0aef9b384116" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab81b56721004d4c986c0aef9b384116" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63eeeb64a5d542de87b1e2511c37bf96" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 32797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60f309e3628a469eb2dcbabae779adfc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38905 } health_report { overall_health: UNKNOWN } } }
Aug 08 10:44:28 dist-test-slave-twwt krb5kdc[1057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.129@KRBTEST.COM
I20260808 10:44:28.070757 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:28.060651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51852 (local address 127.31.237.129:45001)
0808 10:44:28.060800 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:28.060807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:28.061048 (+ 241us) server_negotiation.cc:407] Connection header received
0808 10:44:28.061211 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:28.061214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:28.061267 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:28.061350 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:28.062175 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:28.062795 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:28.063820 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:28.064035 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:28.064118 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:28.066899 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:28.066909 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:28.066925 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:28.066948 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:28.069029 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:28.069443 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:28.069449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:28.069450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:28.069533 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:28.069767 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:28.069769 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:28.069770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:28.070031 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:28.070219 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:28.070341 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 10:44:28.070439 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 10:44:28 dist-test-slave-twwt krb5kdc[1057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185868, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:44:28 dist-test-slave-twwt krb5kdc[1057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185868, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.237.129@KRBTEST.COM
I20260808 10:44:28.100996 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:28.091533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51860 (local address 127.31.237.129:45001)
0808 10:44:28.091697 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:28.091702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:28.091856 (+ 154us) server_negotiation.cc:407] Connection header received
0808 10:44:28.092011 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:28.092014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:28.092058 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:28.092130 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:28.092955 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:28.093446 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:28.094600 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:28.094972 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:28.095056 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:28.097840 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:28.097843 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:28.097845 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:28.097867 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:28.099351 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:28.099786 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:28.099788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:28.099789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:28.099860 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:28.100128 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:28.100130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:28.100131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:28.100281 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:28.100497 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:28.100697 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 10:44:28.100795 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 10:44:28 dist-test-slave-twwt krb5kdc[1057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:44:28 dist-test-slave-twwt krb5kdc[1057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.129@KRBTEST.COM
I20260808 10:44:28.130501 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:28.122326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51876 (local address 127.31.237.129:45001)
0808 10:44:28.122534 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:28.122538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:28.122595 (+ 57us) server_negotiation.cc:407] Connection header received
0808 10:44:28.122771 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:28.122773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:28.122821 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:28.122893 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:28.123521 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:28.124020 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:28.124810 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:28.125063 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:28.125185 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:28.127668 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:28.127672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:28.127674 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:28.127698 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:28.129152 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:28.129532 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:28.129536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:28.129537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:28.129608 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:28.129833 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:28.129835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:28.129836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:28.130006 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:28.130122 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:28.130278 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 10:44:28.130372 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260808 10:44:28.131976 1283 tablet_service.cc:2908] Rejecting scan request for tablet 8cf4b22266b94d7da2db1f6ca6229079: Uninitialized: clean time has not yet been initialized
I20260808 10:44:28.134100 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 1219
W20260808 10:44:28.149468 1826 tablet.cc:2369] T 8cf4b22266b94d7da2db1f6ca6229079 P 63eeeb64a5d542de87b1e2511c37bf96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:44:28.150131 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 1428
I20260808 10:44:28.165864 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 1628
I20260808 10:44:28.181548 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 1075
2026-08-08T10:44:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2697 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:28 dist-test-slave-twwt krb5kdc[1891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:28 dist-test-slave-twwt krb5kdc[1891](info): set up 2 sockets
Aug 08 10:44:28 dist-test-slave-twwt krb5kdc[1891](info): commencing operation
krb5kdc: starting...
W20260808 10:44:29.425014 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:44:29 dist-test-slave-twwt krb5kdc[1891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:44:29.558710 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:39009
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:39795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:39009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:29.676816 1912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:29.677150 1912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:29.677222 1912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:29.681378 1912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:29.681466 1912 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:44:29.681499 1912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:29.681559 1912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:29.681614 1912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:29.681650 1912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:29.687270 1912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:39795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:39009
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:39009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:29.688378 1912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:29.689428 1912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:29.698113 1925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:29.698148 1921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:29.698173 1922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:29.698307 1912 server_base.cc:1034] running on GCE node
I20260808 10:44:29.699131 1912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:29.700225 1912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:29.701395 1912 hybrid_clock.cc:648] HybridClock initialized: now 1786185869701382 us; error 26 us; skew 500 ppm
Aug 08 10:44:29 dist-test-slave-twwt krb5kdc[1891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185869, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:29.704398 1912 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:44:29.705366 1912 webserver.cc:460] Webserver started at http://127.31.237.190:35839/ using document root <none> and password file <none>
I20260808 10:44:29.706010 1912 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:29.706076 1912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:29.706305 1912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:29.708446 1912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "2240830aaf3c48b5bbd10f5c0ec9c508"
format_stamp: "Formatted at 2026-08-08 10:44:29 on dist-test-slave-twwt"
server_key: "345f5e50c120030d74e846e4cc36a0ce"
server_key_iv: "e33072f1ba350ed04de58a54e7468f91"
server_key_version: "encryptionkey@0"
I20260808 10:44:29.709045 1912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "2240830aaf3c48b5bbd10f5c0ec9c508"
format_stamp: "Formatted at 2026-08-08 10:44:29 on dist-test-slave-twwt"
server_key: "345f5e50c120030d74e846e4cc36a0ce"
server_key_iv: "e33072f1ba350ed04de58a54e7468f91"
server_key_version: "encryptionkey@0"
I20260808 10:44:29.712925 1912 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:44:29.715575 1939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:29.716986 1912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:29.717150 1912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "2240830aaf3c48b5bbd10f5c0ec9c508"
format_stamp: "Formatted at 2026-08-08 10:44:29 on dist-test-slave-twwt"
server_key: "345f5e50c120030d74e846e4cc36a0ce"
server_key_iv: "e33072f1ba350ed04de58a54e7468f91"
server_key_version: "encryptionkey@0"
I20260808 10:44:29.717271 1912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:29.734726 1912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:29.737951 1912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:29.738150 1912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:29.746277 1912 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:39009
I20260808 10:44:29.746328 2031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:39009 every 8 connection(s)
I20260808 10:44:29.747251 1912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:29.750046 2036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:29.755743 2036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508: Bootstrap starting.
I20260808 10:44:29.756497 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 1912
I20260808 10:44:29.756651 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:29.756928 32694 external_mini_cluster.cc:1428] Setting key 1e75747aeb0a29275ec26ccee61c8ae4
I20260808 10:44:29.758934 2036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:29.759704 2036 log.cc:824] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:29.761664 2036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508: No bootstrap required, opened a new log
I20260808 10:44:29.765691 2036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2240830aaf3c48b5bbd10f5c0ec9c508" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39009 } }
Aug 08 10:44:29 dist-test-slave-twwt krb5kdc[1891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:29.766052 2036 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:29.766139 2036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2240830aaf3c48b5bbd10f5c0ec9c508, State: Initialized, Role: FOLLOWER
I20260808 10:44:29.766727 2036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2240830aaf3c48b5bbd10f5c0ec9c508" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39009 } }
I20260808 10:44:29.766871 2036 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:29.766983 2036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:29.767138 2036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:29.768108 2036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2240830aaf3c48b5bbd10f5c0ec9c508" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39009 } }
I20260808 10:44:29.768496 2036 leader_election.cc:302] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2240830aaf3c48b5bbd10f5c0ec9c508; no voters:
I20260808 10:44:29.768847 2036 leader_election.cc:288] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:29.768980 2046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:29.769227 2046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [term 1 LEADER]: Becoming Leader. State: Replica: 2240830aaf3c48b5bbd10f5c0ec9c508, State: Running, Role: LEADER
I20260808 10:44:29.769666 2046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2240830aaf3c48b5bbd10f5c0ec9c508" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39009 } }
I20260808 10:44:29.770390 2036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:29.771710 2047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2240830aaf3c48b5bbd10f5c0ec9c508" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2240830aaf3c48b5bbd10f5c0ec9c508" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39009 } } }
I20260808 10:44:29.771811 2049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2240830aaf3c48b5bbd10f5c0ec9c508. Latest consensus state: current_term: 1 leader_uuid: "2240830aaf3c48b5bbd10f5c0ec9c508" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2240830aaf3c48b5bbd10f5c0ec9c508" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39009 } } }
I20260808 10:44:29.771970 2049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:29.771970 2047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:29.773520 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:29.758303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50022 (local address 127.31.237.190:39009)
0808 10:44:29.758854 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:29.758870 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:29.758905 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:44:29.759619 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:29.759637 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:29.760050 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:29.760401 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:29.761227 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:29.762254 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:29.763183 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:29.763671 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:29.763857 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:29.766762 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:29.766767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:29.766783 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:29.766813 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:29.769099 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:29.769667 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:29.769675 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:29.769681 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:29.769832 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:29.770285 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:29.770289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:29.770290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:29.770913 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:29.771838 (+ 925us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:29.772005 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 10:44:29.772390 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":195,"threads_started":1}
W20260808 10:44:29.775837 2072 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:29.775921 2072 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:29.775980 2063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:29.776678 2063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:29.782032 2063 catalog_manager.cc:1347] Generated new cluster ID: be768f7b144f458fb903569807862627
I20260808 10:44:29.782101 2063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:29.800752 2063 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:29.801692 2063 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:29.809101 2063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508: Generated new TSK 0
I20260808 10:44:29.809784 2063 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:44:29.846225 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39009
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:39795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:29.998322 2083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:29.998590 2083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:29.998656 2083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:30.002548 2083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:30.002620 2083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:30.002662 2083 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:44:30.002693 2083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:44:30.007884 2083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:39795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.237.190:39009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:30.008971 2083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:30.009987 2083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:30.020105 2089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:30.020089 2088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:30.020294 2083 server_base.cc:1034] running on GCE node
W20260808 10:44:30.020108 2092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:30.020846 2083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:30.021688 2083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:30.022881 2083 hybrid_clock.cc:648] HybridClock initialized: now 1786185870022844 us; error 51 us; skew 500 ppm
Aug 08 10:44:30 dist-test-slave-twwt krb5kdc[1891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185870, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:30.025970 2083 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:44:30.027109 2083 webserver.cc:460] Webserver started at http://127.31.237.129:33727/ using document root <none> and password file <none>
I20260808 10:44:30.027719 2083 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:30.027827 2083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:30.028072 2083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:30.030280 2083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "1a90113af6ef46d6984d9ca704f3159a"
format_stamp: "Formatted at 2026-08-08 10:44:30 on dist-test-slave-twwt"
server_key: "286aec011aaf20ab8136fc0e480b29fd"
server_key_iv: "2e99829787c4f90be3c0e60b393efc1d"
server_key_version: "encryptionkey@0"
I20260808 10:44:30.030936 2083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "1a90113af6ef46d6984d9ca704f3159a"
format_stamp: "Formatted at 2026-08-08 10:44:30 on dist-test-slave-twwt"
server_key: "286aec011aaf20ab8136fc0e480b29fd"
server_key_iv: "2e99829787c4f90be3c0e60b393efc1d"
server_key_version: "encryptionkey@0"
I20260808 10:44:30.034940 2083 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:44:30.037488 2102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:30.038756 2083 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:44:30.038898 2083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "1a90113af6ef46d6984d9ca704f3159a"
format_stamp: "Formatted at 2026-08-08 10:44:30 on dist-test-slave-twwt"
server_key: "286aec011aaf20ab8136fc0e480b29fd"
server_key_iv: "2e99829787c4f90be3c0e60b393efc1d"
server_key_version: "encryptionkey@0"
I20260808 10:44:30.039018 2083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:30.049391 2083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:30.052305 2083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:30.052518 2083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:30.054069 2083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:30.054145 2083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:30.054252 2083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:30.054282 2083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:44:30 dist-test-slave-twwt krb5kdc[1891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185870, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:30.065855 2083 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:44811
I20260808 10:44:30.066794 2307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:44811 every 8 connection(s)
I20260808 10:44:30.067090 2083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:44:30.067245 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 2083
I20260808 10:44:30.067378 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:44:30.067725 32694 external_mini_cluster.cc:1428] Setting key 0240c62b30850a81ab1cd624622103d7
I20260808 10:44:30.069489 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.054734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:42715 (local address 127.31.237.190:39009)
0808 10:44:30.054898 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:30.054906 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:30.056249 (+ 1343us) server_negotiation.cc:407] Connection header received
0808 10:44:30.057411 (+ 1162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:30.057417 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:30.057489 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:30.057599 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:30.059028 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.059936 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.061491 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.061746 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.061867 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.065163 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:30.065167 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:30.065170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:30.065202 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:30.066935 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:30.067583 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:30.067586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:30.067589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:30.067689 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:30.068130 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:30.068137 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:30.068140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:30.068361 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:30.068478 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:30.069127 (+ 649us) server_negotiation.cc:299] Negotiation successful
0808 10:44:30.069312 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:44:30.070312 2114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.055633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39009 (local address 127.31.237.129:42715)
0808 10:44:30.056076 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:30.056127 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:30.057197 (+ 1070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:30.057798 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:30.057805 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:30.058172 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:30.058863 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.058880 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.060089 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.060093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.061347 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.061358 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.061930 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.061934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.062182 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.062879 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:30.062908 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:30.064968 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:30.067046 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:30.067055 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:30.067072 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:30.067444 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:30.067810 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:30.067813 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:30.067816 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:30.068009 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:30.068480 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:30.068488 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:30.068784 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 10:44:30.068982 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 10:44:30.069276 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":126,"threads_started":1}
I20260808 10:44:30.071926 2309 heartbeater.cc:347] Connected to a master server at 127.31.237.190:39009
I20260808 10:44:30.072077 2309 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:30.072613 2309 heartbeater.cc:511] Master 127.31.237.190:39009 requested a full tablet report, sending...
I20260808 10:44:30.074415 1966 ts_manager.cc:194] Registered new tserver with Master: 1a90113af6ef46d6984d9ca704f3159a (127.31.237.129:44811)
I20260808 10:44:30.076110 1966 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:42715
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:44:30.082298 2318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.074384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39009 (local address 127.31.237.129:36549)
0808 10:44:30.074727 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:30.074739 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:30.074860 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:30.075099 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:30.075102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:30.075320 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:30.075417 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.075423 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.076227 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.076230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.076958 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.076968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.077837 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.077843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.077970 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.078539 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:30.078559 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:30.079057 (+ 498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:30.080792 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:30.080796 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:30.080798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:30.081060 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:30.081354 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:30.081358 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:30.081360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:30.081450 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:30.081859 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:30.081862 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:30.081954 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 10:44:30.082041 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:44:30.082150 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":110,"threads_started":1}
I20260808 10:44:30.082316 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.074431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:36549 (local address 127.31.237.190:39009)
0808 10:44:30.074555 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:30.074560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:30.074850 (+ 290us) server_negotiation.cc:407] Connection header received
0808 10:44:30.074952 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:30.074956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:30.074998 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:30.075083 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:30.075540 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.076098 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.077127 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.077559 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.077687 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.079189 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:30.079192 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:30.079194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:30.079217 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:30.080663 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:30.081168 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:30.081177 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:30.081178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:30.081250 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:30.081542 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:30.081546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:30.081548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:30.081715 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:30.081840 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:30.082107 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 10:44:30.082203 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:44:30.101004 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39009
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:39795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:30.216256 2320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:30.216513 2320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:30.216586 2320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:30.220680 2320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:30.220742 2320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:30.220906 2320 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:44:30.220975 2320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:44:30.226228 2320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:39795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.237.190:39009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:30.227350 2320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:30.228382 2320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:30.238185 2326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:30.238368 2332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:30.238457 2330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:30.238725 2320 server_base.cc:1034] running on GCE node
I20260808 10:44:30.239162 2320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:30.239832 2320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:30.241020 2320 hybrid_clock.cc:648] HybridClock initialized: now 1786185870240990 us; error 42 us; skew 500 ppm
Aug 08 10:44:30 dist-test-slave-twwt krb5kdc[1891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185870, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:30.244164 2320 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:44:30.245019 2320 webserver.cc:460] Webserver started at http://127.31.237.130:38979/ using document root <none> and password file <none>
I20260808 10:44:30.245568 2320 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:30.245627 2320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:30.245795 2320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:30.247936 2320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "7cc46fe6e60241889217d1a1466fb892"
format_stamp: "Formatted at 2026-08-08 10:44:30 on dist-test-slave-twwt"
server_key: "6f1267d21ba44fe957524a22edcb63cd"
server_key_iv: "33caa3ec7efd23fe9ed8311b4664bc95"
server_key_version: "encryptionkey@0"
I20260808 10:44:30.248489 2320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "7cc46fe6e60241889217d1a1466fb892"
format_stamp: "Formatted at 2026-08-08 10:44:30 on dist-test-slave-twwt"
server_key: "6f1267d21ba44fe957524a22edcb63cd"
server_key_iv: "33caa3ec7efd23fe9ed8311b4664bc95"
server_key_version: "encryptionkey@0"
I20260808 10:44:30.252470 2320 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:44:30.255059 2343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:30.256346 2320 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:44:30.256500 2320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "7cc46fe6e60241889217d1a1466fb892"
format_stamp: "Formatted at 2026-08-08 10:44:30 on dist-test-slave-twwt"
server_key: "6f1267d21ba44fe957524a22edcb63cd"
server_key_iv: "33caa3ec7efd23fe9ed8311b4664bc95"
server_key_version: "encryptionkey@0"
I20260808 10:44:30.256623 2320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:30.273826 2320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:30.277400 2320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:30.277640 2320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:30.279296 2320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:30.279373 2320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:30.279462 2320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:30.279541 2320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 08 10:44:30 dist-test-slave-twwt krb5kdc[1891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185870, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:30.290905 2320 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:41361
I20260808 10:44:30.290944 2519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:41361 every 8 connection(s)
I20260808 10:44:30.292104 2320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:44:30.294955 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.280063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:48419 (local address 127.31.237.190:39009)
0808 10:44:30.280199 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:30.280203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:30.282058 (+ 1855us) server_negotiation.cc:407] Connection header received
0808 10:44:30.283460 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:30.283464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:30.283527 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:30.283616 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:30.285462 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.286335 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.287719 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.287970 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.288085 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.291132 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:30.291135 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:30.291138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:30.291166 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:30.292783 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:30.293342 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:30.293346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:30.293348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:30.293443 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:30.293819 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:30.293822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:30.293824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:30.294024 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:30.294132 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:30.294660 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 10:44:30.294784 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:44:30.295738 2352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.281292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39009 (local address 127.31.237.130:48419)
0808 10:44:30.281892 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:30.281939 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:30.283191 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:30.283869 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:30.283880 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:30.284452 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:30.285271 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.285296 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.286464 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.286468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.287585 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.287596 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.288133 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.288137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.288382 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.289120 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:30.289145 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:30.290948 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:30.292937 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:30.292946 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:30.292962 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:30.293224 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:30.293546 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:30.293549 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:30.293551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:30.293711 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:30.294132 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:30.294139 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:30.294381 (+ 242us) client_negotiation.cc:769] Sending connection context
0808 10:44:30.294565 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 10:44:30.294903 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":151,"threads_started":1}
I20260808 10:44:30.297061 2520 heartbeater.cc:347] Connected to a master server at 127.31.237.190:39009
I20260808 10:44:30.297269 2520 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:30.297777 2520 heartbeater.cc:511] Master 127.31.237.190:39009 requested a full tablet report, sending...
I20260808 10:44:30.298779 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 2320
I20260808 10:44:30.298900 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:44:30.298979 1966 ts_manager.cc:194] Registered new tserver with Master: 7cc46fe6e60241889217d1a1466fb892 (127.31.237.130:41361)
I20260808 10:44:30.299203 32694 external_mini_cluster.cc:1428] Setting key 45384df8318e65c37d786008c7e149e7
I20260808 10:44:30.299774 1966 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:48419
I20260808 10:44:30.307065 2527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.299277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39009 (local address 127.31.237.130:43585)
0808 10:44:30.299642 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:30.299656 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:30.299753 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:30.300033 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:30.300036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:30.300245 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:30.300367 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.300375 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.301245 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.301247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.301792 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.301798 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.302249 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.302252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.302350 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.302946 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:30.302968 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:30.303421 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:30.305445 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:30.305448 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:30.305450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:30.305634 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:30.305935 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:30.305937 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:30.305938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:30.306012 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:30.306590 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:30.306595 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:30.306720 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 10:44:30.306810 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 10:44:30.306916 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":147,"threads_started":1}
I20260808 10:44:30.307204 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.299368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:43585 (local address 127.31.237.190:39009)
0808 10:44:30.299478 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:30.299482 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:30.299715 (+ 233us) server_negotiation.cc:407] Connection header received
0808 10:44:30.299838 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:30.299840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:30.299909 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:30.300014 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:30.300520 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.301112 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.301917 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.302130 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.302234 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.303572 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:30.303575 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:30.303577 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:30.303598 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:30.305316 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:30.305731 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:30.305737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:30.305742 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:30.305839 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:30.306108 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:30.306114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:30.306119 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:30.306463 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:30.306590 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:30.306905 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 10:44:30.307031 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:44:30.329783 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39009
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:39795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:30.446600 2531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:30.446946 2531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:30.447057 2531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:30.451116 2531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:30.451201 2531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:30.451270 2531 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:44:30.451355 2531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:44:30.456753 2531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:39795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.237.190:39009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:30.457896 2531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:30.459046 2531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:30.469417 2549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:30.469412 2536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:30.469412 2537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:30.469666 2531 server_base.cc:1034] running on GCE node
I20260808 10:44:30.470207 2531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:30.471072 2531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:30.472266 2531 hybrid_clock.cc:648] HybridClock initialized: now 1786185870472228 us; error 53 us; skew 500 ppm
Aug 08 10:44:30 dist-test-slave-twwt krb5kdc[1891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185870, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:30.479669 2531 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:44:30.480726 2531 webserver.cc:460] Webserver started at http://127.31.237.131:40813/ using document root <none> and password file <none>
I20260808 10:44:30.481334 2531 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:30.481427 2531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:30.481724 2531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:30.484012 2531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "26970278bb2441df8e19be77ee6ff1be"
format_stamp: "Formatted at 2026-08-08 10:44:30 on dist-test-slave-twwt"
server_key: "3dcb7a477f623fd41ee8b3b7c995d404"
server_key_iv: "e23de9c51a1740f92943b406f149f1a2"
server_key_version: "encryptionkey@0"
I20260808 10:44:30.484603 2531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "26970278bb2441df8e19be77ee6ff1be"
format_stamp: "Formatted at 2026-08-08 10:44:30 on dist-test-slave-twwt"
server_key: "3dcb7a477f623fd41ee8b3b7c995d404"
server_key_iv: "e23de9c51a1740f92943b406f149f1a2"
server_key_version: "encryptionkey@0"
I20260808 10:44:30.488711 2531 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:44:30.491569 2558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:30.492909 2531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:30.493055 2531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "26970278bb2441df8e19be77ee6ff1be"
format_stamp: "Formatted at 2026-08-08 10:44:30 on dist-test-slave-twwt"
server_key: "3dcb7a477f623fd41ee8b3b7c995d404"
server_key_iv: "e23de9c51a1740f92943b406f149f1a2"
server_key_version: "encryptionkey@0"
I20260808 10:44:30.493178 2531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:30.506870 2531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:30.509799 2531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:30.510007 2531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:30.511634 2531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:30.511711 2531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:30.511785 2531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:30.511837 2531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:44:30 dist-test-slave-twwt krb5kdc[1891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185870, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:30.525127 2531 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:45279
I20260808 10:44:30.525200 2735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:45279 every 8 connection(s)
I20260808 10:44:30.525682 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.512393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:55031 (local address 127.31.237.190:39009)
0808 10:44:30.512513 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:30.512517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:30.513582 (+ 1065us) server_negotiation.cc:407] Connection header received
0808 10:44:30.514477 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:30.514481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:30.514538 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:30.514662 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:30.515976 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.516539 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.517700 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.517973 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.518091 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.521683 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:30.521686 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:30.521689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:30.521717 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:30.523418 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:30.524004 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:30.524009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:30.524013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:30.524096 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:30.524435 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:30.524451 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:30.524455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:30.524700 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:30.524805 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:30.525464 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 10:44:30.525567 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 10:44:30.526188 2531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:44:30.526786 2569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.512946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39009 (local address 127.31.237.131:55031)
0808 10:44:30.513437 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:30.513478 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:30.514267 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:30.514852 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:30.514862 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:30.515351 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:30.515824 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.515837 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.516695 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.516698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.517568 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.517578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.518134 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.518139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.518302 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.519019 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:30.519035 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:30.521470 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:30.523555 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:30.523563 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:30.523580 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:30.523894 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:30.524183 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:30.524189 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:30.524193 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:30.524344 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:30.524866 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:30.524873 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:30.525203 (+ 330us) client_negotiation.cc:769] Sending connection context
0808 10:44:30.525405 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 10:44:30.525732 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":140,"threads_started":1}
I20260808 10:44:30.527089 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 2531
I20260808 10:44:30.527220 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:44:30.527559 32694 external_mini_cluster.cc:1428] Setting key 17e1506d554815fe34c2999de3bffe2e
I20260808 10:44:30.528309 2736 heartbeater.cc:347] Connected to a master server at 127.31.237.190:39009
I20260808 10:44:30.528631 2736 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:30.529222 2736 heartbeater.cc:511] Master 127.31.237.190:39009 requested a full tablet report, sending...
I20260808 10:44:30.530303 1966 ts_manager.cc:194] Registered new tserver with Master: 26970278bb2441df8e19be77ee6ff1be (127.31.237.131:45279)
I20260808 10:44:30.531112 1966 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:55031
I20260808 10:44:30.537474 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.530594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39009 (local address 127.31.237.131:42983)
0808 10:44:30.530984 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:30.530997 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:30.531105 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:30.531312 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:30.531314 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:30.531527 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:30.531620 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.531626 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.532397 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.532401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.533222 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.533231 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.533628 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.533631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.533695 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.534117 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:30.534127 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:30.534419 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:30.536176 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:30.536179 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:30.536180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:30.536365 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:30.536618 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:30.536620 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:30.536621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:30.536696 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:30.537038 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:30.537041 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:30.537145 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 10:44:30.537240 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 10:44:30.537344 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":200,"threads_started":1}
I20260808 10:44:30.537503 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.530715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:42983 (local address 127.31.237.190:39009)
0808 10:44:30.530823 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:30.530827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:30.531066 (+ 239us) server_negotiation.cc:407] Connection header received
0808 10:44:30.531181 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:30.531184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:30.531226 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:30.531297 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:30.531784 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.532262 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.533327 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.533535 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.533611 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.534551 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:30.534554 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:30.534556 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:30.534579 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:30.536076 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:30.536459 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:30.536462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:30.536463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:30.536536 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:30.536774 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:30.536776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:30.536777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:30.536942 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:30.537019 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:30.537295 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 10:44:30.537384 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:44:30.541424 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:44:30 dist-test-slave-twwt krb5kdc[1891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185870, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:44:30 dist-test-slave-twwt krb5kdc[1891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185870, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:30.570261 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.560543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.31.237.190:39009)
0808 10:44:30.560718 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:30.560723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:30.560738 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:44:30.560788 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:30.560791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:30.560836 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:30.560917 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:30.561934 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.562481 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.563453 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.563708 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.563874 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.566694 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:30.566699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:30.566701 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:30.566731 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:30.568548 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:30.568969 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:30.568972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:30.568974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:30.569064 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:30.569316 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:30.569319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:30.569320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:30.569508 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:30.569614 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:30.569866 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 10:44:30.570062 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:44:30.573732 1966 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:44:30.576962 1966 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:44:30.592468 2769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.586222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44018 (local address 127.31.237.130:41361)
0808 10:44:30.586572 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:30.586584 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:30.587113 (+ 529us) server_negotiation.cc:407] Connection header received
0808 10:44:30.587205 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:30.587213 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:30.587446 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:30.587613 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:30.588320 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.589404 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.591181 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.591753 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.591915 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.592063 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:30.592198 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 10:44:30.592298 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":106,"threads_started":1}
I20260808 10:44:30.592456 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.586211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:44811 (local address 127.0.0.1:49488)
0808 10:44:30.587053 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:30.587066 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:30.587144 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:30.587891 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:30.587896 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:30.587917 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:30.588053 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.588061 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.589442 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.589445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.590884 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.590894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.592010 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.592014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.592099 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.592123 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:44:30.592179 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 10:44:30.592314 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"mutex_wait_us":178,"thread_start_us":123,"threads_started":1}
I20260808 10:44:30.592707 2772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.586053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:41361 (local address 127.0.0.1:44018)
0808 10:44:30.586742 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:30.586799 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:30.586954 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:30.587809 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:30.587817 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:30.587839 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:30.588206 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.588211 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.589496 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.589500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.591024 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.591037 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.591891 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.591897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.592006 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.592044 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 10:44:30.592167 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 10:44:30.592572 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":128,"thread_start_us":69,"threads_started":1}
I20260808 10:44:30.592772 2776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.587156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49488 (local address 127.31.237.129:44811)
0808 10:44:30.587411 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:30.587418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:30.587447 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:44:30.587537 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:30.587542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:30.587734 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:30.587992 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:30.588147 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.589307 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.591023 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.591853 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.592305 (+ 452us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.592455 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:30.592506 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 10:44:30.592603 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":89,"threads_started":1}
I20260808 10:44:30.592976 2773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.586837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:45279 (local address 127.0.0.1:37328)
0808 10:44:30.587607 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:30.587620 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:30.587695 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:30.588389 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:30.588392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:30.588410 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:30.588496 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.588503 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.589506 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.589509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.591024 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.591037 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.592219 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.592222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.592315 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.592334 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:44:30.592402 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:44:30.592476 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":161,"thread_start_us":109,"threads_started":1}
I20260808 10:44:30.593554 2775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.587006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37328 (local address 127.31.237.131:45279)
0808 10:44:30.587958 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:30.587971 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:30.587996 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:44:30.588071 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:30.588076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:30.588272 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:30.588426 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:30.588612 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.589343 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.591149 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.591860 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.592018 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.592156 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:30.593311 (+ 1155us) server_negotiation.cc:299] Negotiation successful
0808 10:44:30.593414 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":843,"thread_start_us":92,"threads_started":1}
I20260808 10:44:30.595773 2415 tablet_service.cc:1467] Processing CreateTablet for tablet c8f1a143658d4b0690c10632ac96d925 (DEFAULT_TABLE table=test-table [id=f0e8b89d8e0b4680929a62929666a4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:30.595799 2195 tablet_service.cc:1467] Processing CreateTablet for tablet c8f1a143658d4b0690c10632ac96d925 (DEFAULT_TABLE table=test-table [id=f0e8b89d8e0b4680929a62929666a4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:30.596869 2415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8f1a143658d4b0690c10632ac96d925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:30.596899 2195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8f1a143658d4b0690c10632ac96d925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:30.597249 2641 tablet_service.cc:1467] Processing CreateTablet for tablet c8f1a143658d4b0690c10632ac96d925 (DEFAULT_TABLE table=test-table [id=f0e8b89d8e0b4680929a62929666a4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:30.598430 2641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8f1a143658d4b0690c10632ac96d925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:30.603945 2777 tablet_bootstrap.cc:492] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892: Bootstrap starting.
I20260808 10:44:30.605535 2778 tablet_bootstrap.cc:492] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a: Bootstrap starting.
I20260808 10:44:30.606155 2777 tablet_bootstrap.cc:654] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:30.606942 2779 tablet_bootstrap.cc:492] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be: Bootstrap starting.
I20260808 10:44:30.606999 2777 log.cc:824] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:30.607724 2778 tablet_bootstrap.cc:654] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:30.608613 2778 log.cc:824] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:30.609072 2779 tablet_bootstrap.cc:654] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:30.609107 2777 tablet_bootstrap.cc:492] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892: No bootstrap required, opened a new log
I20260808 10:44:30.609311 2777 ts_tablet_manager.cc:1397] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:44:30.609931 2779 log.cc:824] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:30.610764 2778 tablet_bootstrap.cc:492] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a: No bootstrap required, opened a new log
I20260808 10:44:30.610988 2778 ts_tablet_manager.cc:1397] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:44:30.612031 2779 tablet_bootstrap.cc:492] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be: No bootstrap required, opened a new log
I20260808 10:44:30.612253 2779 ts_tablet_manager.cc:1397] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:44:30.612860 2777 raft_consensus.cc:348] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.613102 2777 raft_consensus.cc:374] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:30.613175 2777 raft_consensus.cc:729] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc46fe6e60241889217d1a1466fb892, State: Initialized, Role: FOLLOWER
I20260808 10:44:30.613653 2777 consensus_queue.cc:260] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.614367 2520 heartbeater.cc:503] Master 127.31.237.190:39009 was elected leader, sending a full tablet report...
I20260808 10:44:30.614756 2777 ts_tablet_manager.cc:1428] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 10:44:30.616215 2779 raft_consensus.cc:348] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.616526 2779 raft_consensus.cc:374] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:30.616616 2779 raft_consensus.cc:729] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26970278bb2441df8e19be77ee6ff1be, State: Initialized, Role: FOLLOWER
I20260808 10:44:30.616819 2778 raft_consensus.cc:348] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.617121 2778 raft_consensus.cc:374] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:30.617075 2779 consensus_queue.cc:260] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.617214 2778 raft_consensus.cc:729] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a90113af6ef46d6984d9ca704f3159a, State: Initialized, Role: FOLLOWER
I20260808 10:44:30.617798 2736 heartbeater.cc:503] Master 127.31.237.190:39009 was elected leader, sending a full tablet report...
I20260808 10:44:30.618147 2779 ts_tablet_manager.cc:1428] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:44:30.618242 2778 consensus_queue.cc:260] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.618896 2309 heartbeater.cc:503] Master 127.31.237.190:39009 was elected leader, sending a full tablet report...
I20260808 10:44:30.619191 2778 ts_tablet_manager.cc:1428] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260808 10:44:30.688131 2791 raft_consensus.cc:482] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:44:30.688396 2791 raft_consensus.cc:504] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.689870 2791 leader_election.cc:288] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a90113af6ef46d6984d9ca704f3159a (127.31.237.129:44811), 26970278bb2441df8e19be77ee6ff1be (127.31.237.131:45279)
I20260808 10:44:30.694489 2352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.690178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:44811 (local address 127.31.237.130:54237)
0808 10:44:30.690318 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:30.690333 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:30.690431 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:30.690772 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:30.690778 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:30.690809 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:30.690943 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.690950 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.692014 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.692016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.693325 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.693337 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.694090 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.694093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.694236 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.694259 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:44:30.694324 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 10:44:30.694378 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 10:44:30.694572 2776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.690236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:54237 (local address 127.31.237.129:44811)
0808 10:44:30.690370 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:30.690375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:30.690390 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:44:30.690565 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:30.690569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:30.690644 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:30.690742 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:30.691103 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.691904 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.693488 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.693923 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.694094 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.694154 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:30.694366 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 10:44:30.694444 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:44:30.695266 2229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f1a143658d4b0690c10632ac96d925" candidate_uuid: "7cc46fe6e60241889217d1a1466fb892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a90113af6ef46d6984d9ca704f3159a" is_pre_election: true
I20260808 10:44:30.695570 2775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.690332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:57935 (local address 127.31.237.131:45279)
0808 10:44:30.690476 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:30.690479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:30.691074 (+ 595us) server_negotiation.cc:407] Connection header received
0808 10:44:30.691184 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:30.691187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:30.691245 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:30.691327 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:30.691565 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.692213 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.694047 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.694972 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.695067 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.695103 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:30.695358 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 10:44:30.695441 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:44:30.695647 2229 raft_consensus.cc:2454] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc46fe6e60241889217d1a1466fb892 in term 0.
I20260808 10:44:30.695829 2795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.690181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:45279 (local address 127.31.237.130:57935)
0808 10:44:30.690966 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:30.690981 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:30.691080 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:30.691342 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:30.691344 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:30.691358 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:30.691460 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.691464 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.692327 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.692331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.693907 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:30.693917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.695107 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.695111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:30.695216 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.695242 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 10:44:30.695326 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:44:30.695397 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"mutex_wait_us":357,"thread_start_us":155,"threads_started":1}
I20260808 10:44:30.696182 2347 leader_election.cc:302] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a90113af6ef46d6984d9ca704f3159a, 7cc46fe6e60241889217d1a1466fb892; no voters:
I20260808 10:44:30.696511 2791 raft_consensus.cc:2781] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:44:30.696513 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f1a143658d4b0690c10632ac96d925" candidate_uuid: "7cc46fe6e60241889217d1a1466fb892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26970278bb2441df8e19be77ee6ff1be" is_pre_election: true
I20260808 10:44:30.696635 2791 raft_consensus.cc:482] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:44:30.696694 2791 raft_consensus.cc:3037] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:30.696830 2673 raft_consensus.cc:2454] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc46fe6e60241889217d1a1466fb892 in term 0.
I20260808 10:44:30.698053 2791 raft_consensus.cc:504] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.698396 2791 leader_election.cc:288] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [CANDIDATE]: Term 1 election: Requested vote from peers 1a90113af6ef46d6984d9ca704f3159a (127.31.237.129:44811), 26970278bb2441df8e19be77ee6ff1be (127.31.237.131:45279)
I20260808 10:44:30.698746 2229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f1a143658d4b0690c10632ac96d925" candidate_uuid: "7cc46fe6e60241889217d1a1466fb892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a90113af6ef46d6984d9ca704f3159a"
I20260808 10:44:30.698792 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f1a143658d4b0690c10632ac96d925" candidate_uuid: "7cc46fe6e60241889217d1a1466fb892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26970278bb2441df8e19be77ee6ff1be"
I20260808 10:44:30.698900 2229 raft_consensus.cc:3037] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:30.698909 2673 raft_consensus.cc:3037] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:30.700030 2229 raft_consensus.cc:2454] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc46fe6e60241889217d1a1466fb892 in term 1.
I20260808 10:44:30.700352 2673 raft_consensus.cc:2454] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc46fe6e60241889217d1a1466fb892 in term 1.
I20260808 10:44:30.700356 2347 leader_election.cc:302] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a90113af6ef46d6984d9ca704f3159a, 7cc46fe6e60241889217d1a1466fb892; no voters:
I20260808 10:44:30.700541 2791 raft_consensus.cc:2781] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:30.700768 2791 raft_consensus.cc:686] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [term 1 LEADER]: Becoming Leader. State: Replica: 7cc46fe6e60241889217d1a1466fb892, State: Running, Role: LEADER
I20260808 10:44:30.701138 2791 consensus_queue.cc:237] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.704586 1966 catalog_manager.cc:5495] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc46fe6e60241889217d1a1466fb892 (127.31.237.130). New cstate: current_term: 1 leader_uuid: "7cc46fe6e60241889217d1a1466fb892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260808 10:44:30.721717 2769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:30.717795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44020 (local address 127.31.237.130:41361)
0808 10:44:30.717964 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:30.717969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:30.717985 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:44:30.718199 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:30.718201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:30.718265 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:30.718376 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:30.718683 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.719196 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.720126 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:30.720373 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:30.720485 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:30.720865 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:30.721283 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:30.721416 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:30.721509 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 10:44:30.721567 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:44:30.727912 2229 raft_consensus.cc:1261] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a [term 1 FOLLOWER]: Refusing update from remote peer 7cc46fe6e60241889217d1a1466fb892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:30.727914 2673 raft_consensus.cc:1261] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be [term 1 FOLLOWER]: Refusing update from remote peer 7cc46fe6e60241889217d1a1466fb892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:30.728682 2791 consensus_queue.cc:1035] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:30.728922 2797 consensus_queue.cc:1035] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:30.737161 2801 mvcc.cc:205] Tried to move back new op lower bound from 7316217326493110272 to 7316217326394871808. Current Snapshot: MvccSnapshot[applied={T|T < 7316217326493110272}]
I20260808 10:44:30.739462 2805 mvcc.cc:205] Tried to move back new op lower bound from 7316217326493110272 to 7316217326394871808. Current Snapshot: MvccSnapshot[applied={T|T < 7316217326493110272 or (T in {7316217326493110272})}]
I20260808 10:44:30.749281 1966 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:30.749470 1966 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:30.752496 1966 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508: Sending DeleteTablet for 3 replicas of tablet c8f1a143658d4b0690c10632ac96d925
I20260808 10:44:30.753434 2641 tablet_service.cc:1514] Processing DeleteTablet for tablet c8f1a143658d4b0690c10632ac96d925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:30 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:37328
I20260808 10:44:30.753549 2415 tablet_service.cc:1514] Processing DeleteTablet for tablet c8f1a143658d4b0690c10632ac96d925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:30 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:44018
I20260808 10:44:30.754444 2195 tablet_service.cc:1514] Processing DeleteTablet for tablet c8f1a143658d4b0690c10632ac96d925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:30 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:49488
I20260808 10:44:30.754531 1966 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:44:30.754976 1966 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:44:30.755575 2819 tablet_replica.cc:330] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a: stopping tablet replica
I20260808 10:44:30.755970 2819 raft_consensus.cc:2229] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:30.756436 2819 raft_consensus.cc:2258] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:30.758234 2819 ts_tablet_manager.cc:1905] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:30.760136 2817 tablet_replica.cc:330] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892: stopping tablet replica
I20260808 10:44:30.760411 2641 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc51406910b409d95f953b598937f69 (DEFAULT_TABLE table=test-table [id=8b2eaa1fd135467ea1cce9db721c34d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:30.760701 2641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc51406910b409d95f953b598937f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:30.761314 2817 raft_consensus.cc:2229] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:44:30.761808 2817 raft_consensus.cc:2258] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:30.762125 2819 ts_tablet_manager.cc:1918] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:44:30.762221 2819 log.cc:1196] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/wal/wals/c8f1a143658d4b0690c10632ac96d925
I20260808 10:44:30.762555 2819 ts_tablet_manager.cc:1939] T c8f1a143658d4b0690c10632ac96d925 P 1a90113af6ef46d6984d9ca704f3159a: Deleting consensus metadata
I20260808 10:44:30.763470 2195 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc51406910b409d95f953b598937f69 (DEFAULT_TABLE table=test-table [id=8b2eaa1fd135467ea1cce9db721c34d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:30.763772 2195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc51406910b409d95f953b598937f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:30.763908 1945 catalog_manager.cc:4841] TS 1a90113af6ef46d6984d9ca704f3159a (127.31.237.129:44811): tablet c8f1a143658d4b0690c10632ac96d925 (table test-table [id=f0e8b89d8e0b4680929a62929666a4ea]) successfully deleted
I20260808 10:44:30.764946 2415 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc51406910b409d95f953b598937f69 (DEFAULT_TABLE table=test-table [id=8b2eaa1fd135467ea1cce9db721c34d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:30.765230 2415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc51406910b409d95f953b598937f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:30.765969 2817 ts_tablet_manager.cc:1905] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:30.766330 2778 tablet_bootstrap.cc:492] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a: Bootstrap starting.
I20260808 10:44:30.767594 2778 tablet_bootstrap.cc:654] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:30.768697 2777 tablet_bootstrap.cc:492] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892: Bootstrap starting.
I20260808 10:44:30.769070 2817 ts_tablet_manager.cc:1918] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:44:30.769167 2817 log.cc:1196] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/c8f1a143658d4b0690c10632ac96d925
I20260808 10:44:30.769502 2817 ts_tablet_manager.cc:1939] T c8f1a143658d4b0690c10632ac96d925 P 7cc46fe6e60241889217d1a1466fb892: Deleting consensus metadata
I20260808 10:44:30.769721 2777 tablet_bootstrap.cc:654] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:30.770008 2816 tablet_replica.cc:330] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be: stopping tablet replica
I20260808 10:44:30.770401 2779 tablet_bootstrap.cc:492] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be: Bootstrap starting.
I20260808 10:44:30.770432 2816 raft_consensus.cc:2229] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:30.770838 2816 raft_consensus.cc:2258] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:30.771476 2779 tablet_bootstrap.cc:654] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:30.772121 2777 tablet_bootstrap.cc:492] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892: No bootstrap required, opened a new log
I20260808 10:44:30.772235 2777 ts_tablet_manager.cc:1397] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 10:44:30.772837 2779 tablet_bootstrap.cc:492] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be: No bootstrap required, opened a new log
I20260808 10:44:30.772920 2779 ts_tablet_manager.cc:1397] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:44:30.772967 2816 ts_tablet_manager.cc:1905] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:30.772959 2777 raft_consensus.cc:348] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.773099 2777 raft_consensus.cc:374] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:30.773144 2777 raft_consensus.cc:729] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc46fe6e60241889217d1a1466fb892, State: Initialized, Role: FOLLOWER
I20260808 10:44:30.773507 2779 raft_consensus.cc:348] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.773638 2779 raft_consensus.cc:374] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:30.773677 2779 raft_consensus.cc:729] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26970278bb2441df8e19be77ee6ff1be, State: Initialized, Role: FOLLOWER
I20260808 10:44:30.773798 2779 consensus_queue.cc:260] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.774081 2779 ts_tablet_manager.cc:1428] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:44:30.774132 2777 consensus_queue.cc:260] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.774418 2777 ts_tablet_manager.cc:1428] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:44:30.774433 1943 catalog_manager.cc:4841] TS 7cc46fe6e60241889217d1a1466fb892 (127.31.237.130:41361): tablet c8f1a143658d4b0690c10632ac96d925 (table test-table [id=f0e8b89d8e0b4680929a62929666a4ea]) successfully deleted
I20260808 10:44:30.775688 2778 tablet_bootstrap.cc:492] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a: No bootstrap required, opened a new log
I20260808 10:44:30.775804 2778 ts_tablet_manager.cc:1397] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260808 10:44:30.776592 2816 ts_tablet_manager.cc:1918] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:44:30.776684 2816 log.cc:1196] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/c8f1a143658d4b0690c10632ac96d925
I20260808 10:44:30.776994 2816 ts_tablet_manager.cc:1939] T c8f1a143658d4b0690c10632ac96d925 P 26970278bb2441df8e19be77ee6ff1be: Deleting consensus metadata
I20260808 10:44:30.776957 2778 raft_consensus.cc:348] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.777108 2778 raft_consensus.cc:374] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:30.777148 2778 raft_consensus.cc:729] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a90113af6ef46d6984d9ca704f3159a, State: Initialized, Role: FOLLOWER
I20260808 10:44:30.777280 2778 consensus_queue.cc:260] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:30.777576 2778 ts_tablet_manager.cc:1428] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:30.778395 1943 catalog_manager.cc:4841] TS 26970278bb2441df8e19be77ee6ff1be (127.31.237.131:45279): tablet c8f1a143658d4b0690c10632ac96d925 (table test-table [id=f0e8b89d8e0b4680929a62929666a4ea]) successfully deleted
W20260808 10:44:30.778527 2738 tablet.cc:2369] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:44:30.793336 2521 tablet.cc:2369] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:44:30.818845 2311 tablet.cc:2369] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:44:31.113519 2793 raft_consensus.cc:482] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:44:31.113706 2793 raft_consensus.cc:504] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:31.114929 2793 leader_election.cc:288] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc46fe6e60241889217d1a1466fb892 (127.31.237.130:41361), 26970278bb2441df8e19be77ee6ff1be (127.31.237.131:45279)
I20260808 10:44:31.119344 2827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:31.115178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:41361 (local address 127.31.237.129:48963)
0808 10:44:31.115489 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:31.115503 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:31.115654 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:31.116010 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:31.116013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:31.116030 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:31.116165 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:31.116190 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.117073 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.117076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:31.118006 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:31.118016 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.118867 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.118871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:31.118986 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:31.119017 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 10:44:31.119090 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:44:31.119169 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":85,"threads_started":1}
I20260808 10:44:31.119381 2769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:31.115286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:48963 (local address 127.31.237.130:41361)
0808 10:44:31.115478 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:31.115483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:31.115596 (+ 113us) server_negotiation.cc:407] Connection header received
0808 10:44:31.115843 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:31.115847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:31.115902 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:31.115996 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:31.116301 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.116960 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.118148 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.118741 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.118843 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:31.118893 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:31.119124 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 10:44:31.119212 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 10:44:31.119491 2775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:31.115803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:50059 (local address 127.31.237.131:45279)
0808 10:44:31.115936 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:31.115941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:31.116120 (+ 179us) server_negotiation.cc:407] Connection header received
0808 10:44:31.116244 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:31.116249 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:31.116313 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:31.116374 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:31.116683 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.117262 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.118449 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.118903 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.119028 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:31.119096 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:31.119278 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 10:44:31.119354 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:44:31.119725 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:31.115653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:45279 (local address 127.31.237.129:50059)
0808 10:44:31.116040 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:31.116052 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:31.116144 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:31.116415 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:31.116418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:31.116437 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:31.116565 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:31.116571 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.117403 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.117406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:31.118317 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:31.118329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.119030 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.119033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:31.119148 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:31.119171 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:44:31.119250 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 10:44:31.119320 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":1,"thread_start_us":181,"threads_started":1}
I20260808 10:44:31.120102 2443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc51406910b409d95f953b598937f69" candidate_uuid: "1a90113af6ef46d6984d9ca704f3159a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc46fe6e60241889217d1a1466fb892" is_pre_election: true
I20260808 10:44:31.120352 2443 raft_consensus.cc:2454] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a90113af6ef46d6984d9ca704f3159a in term 0.
I20260808 10:44:31.120747 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc51406910b409d95f953b598937f69" candidate_uuid: "1a90113af6ef46d6984d9ca704f3159a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26970278bb2441df8e19be77ee6ff1be" is_pre_election: true
I20260808 10:44:31.120913 2673 raft_consensus.cc:2454] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a90113af6ef46d6984d9ca704f3159a in term 0.
I20260808 10:44:31.120857 2105 leader_election.cc:302] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a90113af6ef46d6984d9ca704f3159a, 7cc46fe6e60241889217d1a1466fb892; no voters:
I20260808 10:44:31.121223 2793 raft_consensus.cc:2781] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:44:31.121281 2793 raft_consensus.cc:482] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:44:31.121308 2793 raft_consensus.cc:3037] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:31.122712 2793 raft_consensus.cc:504] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:31.123082 2793 leader_election.cc:288] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [CANDIDATE]: Term 1 election: Requested vote from peers 7cc46fe6e60241889217d1a1466fb892 (127.31.237.130:41361), 26970278bb2441df8e19be77ee6ff1be (127.31.237.131:45279)
I20260808 10:44:31.123538 2443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc51406910b409d95f953b598937f69" candidate_uuid: "1a90113af6ef46d6984d9ca704f3159a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc46fe6e60241889217d1a1466fb892"
I20260808 10:44:31.123656 2443 raft_consensus.cc:3037] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:31.123616 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc51406910b409d95f953b598937f69" candidate_uuid: "1a90113af6ef46d6984d9ca704f3159a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26970278bb2441df8e19be77ee6ff1be"
I20260808 10:44:31.123797 2673 raft_consensus.cc:3037] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:31.124675 2443 raft_consensus.cc:2454] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a90113af6ef46d6984d9ca704f3159a in term 1.
I20260808 10:44:31.124899 2673 raft_consensus.cc:2454] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a90113af6ef46d6984d9ca704f3159a in term 1.
I20260808 10:44:31.124987 2105 leader_election.cc:302] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a90113af6ef46d6984d9ca704f3159a, 7cc46fe6e60241889217d1a1466fb892; no voters:
I20260808 10:44:31.125193 2793 raft_consensus.cc:2781] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:31.125388 2793 raft_consensus.cc:686] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [term 1 LEADER]: Becoming Leader. State: Replica: 1a90113af6ef46d6984d9ca704f3159a, State: Running, Role: LEADER
I20260808 10:44:31.125663 2793 consensus_queue.cc:237] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:31.127970 1965 catalog_manager.cc:5495] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a90113af6ef46d6984d9ca704f3159a (127.31.237.129). New cstate: current_term: 1 leader_uuid: "1a90113af6ef46d6984d9ca704f3159a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
Aug 08 10:44:31 dist-test-slave-twwt krb5kdc[1891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185869, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:31.182675 2841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:31.172876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39009 (local address 127.0.0.1:50040)
0808 10:44:31.173574 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:31.173592 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:31.173816 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:31.174114 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:31.174117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:31.174575 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:31.174696 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:31.174704 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.175559 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.175562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:31.176110 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:31.176118 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.176570 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.176573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:31.176649 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:31.177372 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:31.177385 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:31.179086 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:31.180924 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:31.180942 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:31.180958 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:31.181198 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:31.181489 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:31.181492 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:31.181494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:31.181574 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:31.181975 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:31.181984 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:31.182096 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 10:44:31.182213 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 10:44:31.182402 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":320,"threads_started":1}
I20260808 10:44:31.182675 2842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:31.172942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50040 (local address 127.31.237.190:39009)
0808 10:44:31.173554 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:31.173622 (+ 68us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:31.173703 (+ 81us) server_negotiation.cc:407] Connection header received
0808 10:44:31.173938 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:31.173942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:31.173992 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:31.174090 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:31.174838 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.175410 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.176242 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.176457 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.176566 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:31.179247 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:31.179251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:31.179253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:31.179274 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:31.180792 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:31.181299 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:31.181305 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:31.181307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:31.181391 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:31.181661 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:31.181666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:31.181668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:31.181853 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:31.181961 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:31.182271 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 10:44:31.182477 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":537,"thread_start_us":328,"threads_started":1}
I20260808 10:44:31.187315 1965 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:50040:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:44:31.188071 1965 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:44:31.193065 2415 tablet_service.cc:1467] Processing CreateTablet for tablet 0f9a4f47c4464a67b9c519700c8a8239 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=738d6508d4a14b6aa6a60d0d28ffee94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:44:31.193040 2195 tablet_service.cc:1467] Processing CreateTablet for tablet 0f9a4f47c4464a67b9c519700c8a8239 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=738d6508d4a14b6aa6a60d0d28ffee94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:44:31.193123 2641 tablet_service.cc:1467] Processing CreateTablet for tablet 0f9a4f47c4464a67b9c519700c8a8239 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=738d6508d4a14b6aa6a60d0d28ffee94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:44:31.193490 2195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f9a4f47c4464a67b9c519700c8a8239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:31.193488 2641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f9a4f47c4464a67b9c519700c8a8239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:31.193511 2415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f9a4f47c4464a67b9c519700c8a8239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:31.196480 2777 tablet_bootstrap.cc:492] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892: Bootstrap starting.
I20260808 10:44:31.196854 2779 tablet_bootstrap.cc:492] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be: Bootstrap starting.
I20260808 10:44:31.197655 2778 tablet_bootstrap.cc:492] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a: Bootstrap starting.
I20260808 10:44:31.197680 2777 tablet_bootstrap.cc:654] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:31.197904 2779 tablet_bootstrap.cc:654] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:31.199069 2777 tablet_bootstrap.cc:492] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892: No bootstrap required, opened a new log
I20260808 10:44:31.199093 2778 tablet_bootstrap.cc:654] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:31.199187 2777 ts_tablet_manager.cc:1397] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:44:31.199421 2779 tablet_bootstrap.cc:492] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be: No bootstrap required, opened a new log
I20260808 10:44:31.199555 2779 ts_tablet_manager.cc:1397] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:44:31.199640 2777 raft_consensus.cc:348] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:31.199802 2777 raft_consensus.cc:374] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:31.199841 2777 raft_consensus.cc:729] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc46fe6e60241889217d1a1466fb892, State: Initialized, Role: FOLLOWER
I20260808 10:44:31.200011 2777 consensus_queue.cc:260] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:31.200281 2779 raft_consensus.cc:348] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:31.200399 2779 raft_consensus.cc:374] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:31.200351 2791 tablet_replica.cc:439] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } } }
I20260808 10:44:31.200455 2779 raft_consensus.cc:729] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26970278bb2441df8e19be77ee6ff1be, State: Initialized, Role: FOLLOWER
I20260808 10:44:31.200510 2791 tablet_replica.cc:442] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:44:31.200618 2779 consensus_queue.cc:260] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:31.200938 2779 ts_tablet_manager.cc:1428] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:44:31.200893 2792 tablet_replica.cc:439] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } } }
I20260808 10:44:31.201042 2792 tablet_replica.cc:442] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:44:31.201462 2777 ts_tablet_manager.cc:1428] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:31.201808 2778 tablet_bootstrap.cc:492] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a: No bootstrap required, opened a new log
I20260808 10:44:31.201924 2778 ts_tablet_manager.cc:1397] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 10:44:31.202502 2778 raft_consensus.cc:348] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:31.202672 2778 raft_consensus.cc:374] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:31.202713 2778 raft_consensus.cc:729] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a90113af6ef46d6984d9ca704f3159a, State: Initialized, Role: FOLLOWER
I20260808 10:44:31.202831 2778 consensus_queue.cc:260] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:31.203394 2793 tablet_replica.cc:439] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } } }
I20260808 10:44:31.203538 2793 tablet_replica.cc:442] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:44:31.203655 2778 ts_tablet_manager.cc:1428] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:44:31.273672 2793 raft_consensus.cc:482] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:44:31.273852 2793 raft_consensus.cc:504] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:31.274340 2793 leader_election.cc:288] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc46fe6e60241889217d1a1466fb892 (127.31.237.130:41361), 26970278bb2441df8e19be77ee6ff1be (127.31.237.131:45279)
I20260808 10:44:31.274938 2443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f9a4f47c4464a67b9c519700c8a8239" candidate_uuid: "1a90113af6ef46d6984d9ca704f3159a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc46fe6e60241889217d1a1466fb892" is_pre_election: true
I20260808 10:44:31.274945 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f9a4f47c4464a67b9c519700c8a8239" candidate_uuid: "1a90113af6ef46d6984d9ca704f3159a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26970278bb2441df8e19be77ee6ff1be" is_pre_election: true
I20260808 10:44:31.275127 2443 raft_consensus.cc:2454] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a90113af6ef46d6984d9ca704f3159a in term 0.
I20260808 10:44:31.275133 2673 raft_consensus.cc:2454] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a90113af6ef46d6984d9ca704f3159a in term 0.
I20260808 10:44:31.275470 2105 leader_election.cc:302] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a90113af6ef46d6984d9ca704f3159a, 7cc46fe6e60241889217d1a1466fb892; no voters:
I20260808 10:44:31.275699 2793 raft_consensus.cc:2781] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:44:31.275760 2793 raft_consensus.cc:482] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:44:31.275787 2793 raft_consensus.cc:3037] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:31.276791 2793 raft_consensus.cc:504] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:31.277123 2793 leader_election.cc:288] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [CANDIDATE]: Term 1 election: Requested vote from peers 7cc46fe6e60241889217d1a1466fb892 (127.31.237.130:41361), 26970278bb2441df8e19be77ee6ff1be (127.31.237.131:45279)
I20260808 10:44:31.277609 2443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f9a4f47c4464a67b9c519700c8a8239" candidate_uuid: "1a90113af6ef46d6984d9ca704f3159a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc46fe6e60241889217d1a1466fb892"
I20260808 10:44:31.277637 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f9a4f47c4464a67b9c519700c8a8239" candidate_uuid: "1a90113af6ef46d6984d9ca704f3159a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26970278bb2441df8e19be77ee6ff1be"
I20260808 10:44:31.277750 2673 raft_consensus.cc:3037] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:31.277761 2443 raft_consensus.cc:3037] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:31.278702 2673 raft_consensus.cc:2454] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a90113af6ef46d6984d9ca704f3159a in term 1.
I20260808 10:44:31.278993 2105 leader_election.cc:302] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a90113af6ef46d6984d9ca704f3159a, 26970278bb2441df8e19be77ee6ff1be; no voters:
I20260808 10:44:31.279021 2443 raft_consensus.cc:2454] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a90113af6ef46d6984d9ca704f3159a in term 1.
I20260808 10:44:31.279299 2793 raft_consensus.cc:2781] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:31.279417 2793 raft_consensus.cc:686] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [term 1 LEADER]: Becoming Leader. State: Replica: 1a90113af6ef46d6984d9ca704f3159a, State: Running, Role: LEADER
I20260808 10:44:31.279577 2793 consensus_queue.cc:237] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } }
I20260808 10:44:31.280282 2829 tablet_replica.cc:439] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a: TxnStatusTablet state changed. Reason: New leader 1a90113af6ef46d6984d9ca704f3159a. Latest consensus state: current_term: 1 leader_uuid: "1a90113af6ef46d6984d9ca704f3159a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } } }
I20260808 10:44:31.280395 2829 tablet_replica.cc:442] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:44:31.280813 2852 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:44:31.281023 2852 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 10:44:31.281427 1967 catalog_manager.cc:5495] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a90113af6ef46d6984d9ca704f3159a (127.31.237.129). New cstate: current_term: 1 leader_uuid: "1a90113af6ef46d6984d9ca704f3159a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260808 10:44:31.307174 2841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:31.301070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:44811 (local address 127.0.0.1:49492)
0808 10:44:31.301331 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:31.301347 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:31.301450 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:31.301827 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:31.301830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:31.301839 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:44:31.301958 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:31.301964 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.304082 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.304086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:31.305108 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:31.305121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.305783 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.305787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:31.305884 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:31.305962 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:44:31.306837 (+ 875us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:44:31.306857 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 10:44:31.306937 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 10:44:31.307027 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260808 10:44:31.307204 2855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:31.301181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49492 (local address 127.31.237.129:44811)
0808 10:44:31.301428 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:31.301433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:31.301451 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:44:31.301585 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:31.301589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:31.301658 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:31.301794 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:31.302092 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.303948 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.305246 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.305655 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.305764 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:31.306125 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:31.306729 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:31.306848 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:31.306949 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 10:44:31.307013 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":104,"threads_started":1}
I20260808 10:44:31.310892 2673 raft_consensus.cc:1261] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be [term 1 FOLLOWER]: Refusing update from remote peer 1a90113af6ef46d6984d9ca704f3159a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:31.310947 2443 raft_consensus.cc:1261] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892 [term 1 FOLLOWER]: Refusing update from remote peer 1a90113af6ef46d6984d9ca704f3159a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:31.311496 2793 consensus_queue.cc:1035] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [LEADER]: Connected to new peer: Peer: permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:31.311686 2829 consensus_queue.cc:1035] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:31.313282 2792 tablet_replica.cc:439] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be: TxnStatusTablet state changed. Reason: New leader 1a90113af6ef46d6984d9ca704f3159a. Latest consensus state: current_term: 1 leader_uuid: "1a90113af6ef46d6984d9ca704f3159a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } } }
I20260808 10:44:31.313395 2792 tablet_replica.cc:442] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:44:31.313603 2791 tablet_replica.cc:439] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892: TxnStatusTablet state changed. Reason: New leader 1a90113af6ef46d6984d9ca704f3159a. Latest consensus state: current_term: 1 leader_uuid: "1a90113af6ef46d6984d9ca704f3159a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } } }
I20260808 10:44:31.313719 2791 tablet_replica.cc:442] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:44:31.317119 2866 tablet_replica.cc:439] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a90113af6ef46d6984d9ca704f3159a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } } }
I20260808 10:44:31.317343 2866 tablet_replica.cc:442] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:44:31.317466 2792 tablet_replica.cc:439] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a90113af6ef46d6984d9ca704f3159a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } } }
I20260808 10:44:31.317590 2792 tablet_replica.cc:442] T 0f9a4f47c4464a67b9c519700c8a8239 P 26970278bb2441df8e19be77ee6ff1be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:44:31.317862 2791 tablet_replica.cc:439] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a90113af6ef46d6984d9ca704f3159a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } } }
I20260808 10:44:31.317977 2791 tablet_replica.cc:442] T 0f9a4f47c4464a67b9c519700c8a8239 P 7cc46fe6e60241889217d1a1466fb892: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:44:31.318161 2866 tablet_replica.cc:439] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a90113af6ef46d6984d9ca704f3159a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a90113af6ef46d6984d9ca704f3159a" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44811 } } peers { permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 } } peers { permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 } } }
I20260808 10:44:31.318320 2866 tablet_replica.cc:442] T 0f9a4f47c4464a67b9c519700c8a8239 P 1a90113af6ef46d6984d9ca704f3159a: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:44:31.318704 2858 mvcc.cc:205] Tried to move back new op lower bound from 7316217328883363840 to 7316217328762724352. Current Snapshot: MvccSnapshot[applied={T|T < 7316217328883363840}]
I20260808 10:44:31.330585 2855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:31.326545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49502 (local address 127.31.237.129:44811)
0808 10:44:31.326800 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:31.326807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:31.326938 (+ 131us) server_negotiation.cc:407] Connection header received
0808 10:44:31.327109 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:31.327112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:31.327155 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:31.327218 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:31.327515 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.328098 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.329402 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.329626 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.329706 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:31.330043 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:31.330111 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:31.330163 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:31.330361 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 10:44:31.330407 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 10:44:31.341764 2855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:31.338451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:59343 (local address 127.31.237.129:44811)
0808 10:44:31.338590 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:31.338594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:31.338807 (+ 213us) server_negotiation.cc:407] Connection header received
0808 10:44:31.338952 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:31.338955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:31.338999 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:31.339073 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:31.339321 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.339800 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.340710 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.340916 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.341022 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:31.341298 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:31.341362 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:31.341443 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:31.341530 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 10:44:31.341578 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 10:44:31.342053 2879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:31.338387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:44811 (local address 127.31.237.129:59343)
0808 10:44:31.338722 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:31.338733 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:31.338854 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:31.339092 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:31.339095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:31.339102 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:44:31.339210 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:31.339216 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.339916 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.339919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:31.340587 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:31.340596 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.341021 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.341025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:31.341113 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:31.341164 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:44:31.341451 (+ 287us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:44:31.341459 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:44:31.341522 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 10:44:31.341601 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":3,"thread_start_us":112,"threads_started":1}
I20260808 10:44:31.349615 2443 raft_consensus.cc:1261] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892 [term 1 FOLLOWER]: Refusing update from remote peer 1a90113af6ef46d6984d9ca704f3159a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:31.349608 2673 raft_consensus.cc:1261] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be [term 1 FOLLOWER]: Refusing update from remote peer 1a90113af6ef46d6984d9ca704f3159a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:31.349998 2866 consensus_queue.cc:1035] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc46fe6e60241889217d1a1466fb892" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 41361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:31.350399 2829 consensus_queue.cc:1035] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [LEADER]: Connected to new peer: Peer: permanent_uuid: "26970278bb2441df8e19be77ee6ff1be" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:31.394179 1965 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:31.394381 1965 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:31.396530 1965 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2240830aaf3c48b5bbd10f5c0ec9c508: Sending DeleteTablet for 3 replicas of tablet 8dc51406910b409d95f953b598937f69
I20260808 10:44:31.397177 2194 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dc51406910b409d95f953b598937f69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:31 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:49488
I20260808 10:44:31.397233 2415 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dc51406910b409d95f953b598937f69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:31 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:44018
I20260808 10:44:31.397228 2641 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dc51406910b409d95f953b598937f69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:31 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:37328
I20260808 10:44:31.397678 2890 tablet_replica.cc:330] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892: stopping tablet replica
I20260808 10:44:31.397821 2890 raft_consensus.cc:2229] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:31.397961 2890 raft_consensus.cc:2258] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:31.398195 2888 tablet_replica.cc:330] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be: stopping tablet replica
I20260808 10:44:31.398326 2888 raft_consensus.cc:2229] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:31.398461 2888 raft_consensus.cc:2258] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:31.398944 2890 ts_tablet_manager.cc:1905] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:31.399041 2885 tablet_replica.cc:330] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a: stopping tablet replica
I20260808 10:44:31.399127 2888 ts_tablet_manager.cc:1905] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:31.399183 2885 raft_consensus.cc:2229] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:44:31.399495 2885 raft_consensus.cc:2258] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:31.400154 2885 ts_tablet_manager.cc:1905] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:31.402347 2888 ts_tablet_manager.cc:1918] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:44:31.402444 2888 log.cc:1196] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/8dc51406910b409d95f953b598937f69
I20260808 10:44:31.402387 2890 ts_tablet_manager.cc:1918] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:44:31.402707 2890 log.cc:1196] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/8dc51406910b409d95f953b598937f69
I20260808 10:44:31.402765 2888 ts_tablet_manager.cc:1939] T 8dc51406910b409d95f953b598937f69 P 26970278bb2441df8e19be77ee6ff1be: Deleting consensus metadata
I20260808 10:44:31.403067 2890 ts_tablet_manager.cc:1939] T 8dc51406910b409d95f953b598937f69 P 7cc46fe6e60241889217d1a1466fb892: Deleting consensus metadata
I20260808 10:44:31.403272 2885 ts_tablet_manager.cc:1918] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:44:31.403375 2885 log.cc:1196] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786185862616645-32694-0/minicluster-data/ts-0/wal/wals/8dc51406910b409d95f953b598937f69
I20260808 10:44:31.403517 1943 catalog_manager.cc:4841] TS 26970278bb2441df8e19be77ee6ff1be (127.31.237.131:45279): tablet 8dc51406910b409d95f953b598937f69 (table test-table [id=8b2eaa1fd135467ea1cce9db721c34d7]) successfully deleted
I20260808 10:44:31.403818 2885 ts_tablet_manager.cc:1939] T 8dc51406910b409d95f953b598937f69 P 1a90113af6ef46d6984d9ca704f3159a: Deleting consensus metadata
I20260808 10:44:31.403988 1943 catalog_manager.cc:4841] TS 7cc46fe6e60241889217d1a1466fb892 (127.31.237.130:41361): tablet 8dc51406910b409d95f953b598937f69 (table test-table [id=8b2eaa1fd135467ea1cce9db721c34d7]) successfully deleted
I20260808 10:44:31.404595 1945 catalog_manager.cc:4841] TS 1a90113af6ef46d6984d9ca704f3159a (127.31.237.129:44811): tablet 8dc51406910b409d95f953b598937f69 (table test-table [id=8b2eaa1fd135467ea1cce9db721c34d7]) successfully deleted
Aug 08 10:44:31 dist-test-slave-twwt krb5kdc[1891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185870, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.237.129@KRBTEST.COM
I20260808 10:44:31.418323 2855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:31.407029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49514 (local address 127.31.237.129:44811)
0808 10:44:31.407173 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:31.407178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:31.407429 (+ 251us) server_negotiation.cc:407] Connection header received
0808 10:44:31.407565 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:31.407567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:31.407607 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:31.407673 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:31.408615 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.409096 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.410528 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.410811 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.410888 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:31.414849 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:31.414860 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:31.414878 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:31.414901 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:31.416549 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:31.417001 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:31.417007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:31.417008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:31.417087 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:31.417370 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:31.417374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:31.417376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:31.417622 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:31.417857 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:31.418016 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 10:44:31.418143 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 10:44:31.418951 2298 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49514
I20260808 10:44:31.429215 2842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:31.421169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50046 (local address 127.31.237.190:39009)
0808 10:44:31.421317 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:31.421321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:31.421452 (+ 131us) server_negotiation.cc:407] Connection header received
0808 10:44:31.421582 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:31.421590 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:31.421633 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:31.421719 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:31.422449 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.423038 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.423856 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:31.424093 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:31.424199 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:31.425680 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:31.425684 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:31.425686 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:31.425711 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:31.427354 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:31.427779 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:31.427783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:31.427785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:31.427873 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:31.428151 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:31.428154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:31.428155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:31.428422 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:31.428569 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:31.428803 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 10:44:31.428937 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 10:44:31.429673 1965 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50046
I20260808 10:44:31.431104 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 2083
I20260808 10:44:31.448310 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 2320
I20260808 10:44:31.460000 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 2531
I20260808 10:44:31.472842 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 1912
2026-08-08T10:44:31Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3292 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:31 dist-test-slave-twwt krb5kdc[2901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:31 dist-test-slave-twwt krb5kdc[2901](info): set up 2 sockets
Aug 08 10:44:31 dist-test-slave-twwt krb5kdc[2901](info): commencing operation
krb5kdc: starting...
W20260808 10:44:32.735069 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:44:32 dist-test-slave-twwt krb5kdc[2901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:44:32.866153 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:44853
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:44853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:32.984932 2920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:32.985262 2920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:32.985344 2920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:32.989591 2920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:32.989692 2920 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:44:32.989724 2920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:32.989770 2920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:32.989805 2920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:32.989861 2920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:32.995424 2920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:44853
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:44853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:32.996591 2920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:32.997669 2920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:33.006518 2927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:33.006486 2929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:33.006678 2920 server_base.cc:1034] running on GCE node
W20260808 10:44:33.006835 2931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:33.007436 2920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:33.008612 2920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:33.009814 2920 hybrid_clock.cc:648] HybridClock initialized: now 1786185873009765 us; error 35 us; skew 500 ppm
Aug 08 10:44:33 dist-test-slave-twwt krb5kdc[2901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185873, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:33.016692 2920 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:44:33.017750 2920 webserver.cc:460] Webserver started at http://127.31.237.190:36321/ using document root <none> and password file <none>
I20260808 10:44:33.018385 2920 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:33.018452 2920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:33.018748 2920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:33.020956 2920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "59579d95f4b64d32a3213b0c05271c6c"
format_stamp: "Formatted at 2026-08-08 10:44:33 on dist-test-slave-twwt"
server_key: "b8a34faf2b39370297a672caf968f9c0"
server_key_iv: "b0ff12b82c514edfb3ec023bb7612766"
server_key_version: "encryptionkey@0"
I20260808 10:44:33.021595 2920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "59579d95f4b64d32a3213b0c05271c6c"
format_stamp: "Formatted at 2026-08-08 10:44:33 on dist-test-slave-twwt"
server_key: "b8a34faf2b39370297a672caf968f9c0"
server_key_iv: "b0ff12b82c514edfb3ec023bb7612766"
server_key_version: "encryptionkey@0"
I20260808 10:44:33.025668 2920 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 10:44:33.028450 2948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:33.029942 2920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:44:33.030117 2920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "59579d95f4b64d32a3213b0c05271c6c"
format_stamp: "Formatted at 2026-08-08 10:44:33 on dist-test-slave-twwt"
server_key: "b8a34faf2b39370297a672caf968f9c0"
server_key_iv: "b0ff12b82c514edfb3ec023bb7612766"
server_key_version: "encryptionkey@0"
I20260808 10:44:33.030252 2920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:33.054072 2920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:33.057286 2920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:33.057505 2920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:33.066181 2920 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:44853
I20260808 10:44:33.066212 3058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:44853 every 8 connection(s)
I20260808 10:44:33.067216 2920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:33.070183 3059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:33.074343 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 2920
I20260808 10:44:33.074478 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:33.074781 32694 external_mini_cluster.cc:1428] Setting key 9289658501131d28bd8c58e0d342d3ea
I20260808 10:44:33.076550 3059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c: Bootstrap starting.
I20260808 10:44:33.078979 3059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:33.079751 3059 log.cc:824] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:33.081670 3059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c: No bootstrap required, opened a new log
Aug 08 10:44:33 dist-test-slave-twwt krb5kdc[2901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:33.085191 3059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59579d95f4b64d32a3213b0c05271c6c" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44853 } }
I20260808 10:44:33.085459 3059 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:33.085525 3059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59579d95f4b64d32a3213b0c05271c6c, State: Initialized, Role: FOLLOWER
I20260808 10:44:33.086047 3059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59579d95f4b64d32a3213b0c05271c6c" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44853 } }
I20260808 10:44:33.086211 3059 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:33.086299 3059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:33.086421 3059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:33.087399 3059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59579d95f4b64d32a3213b0c05271c6c" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44853 } }
I20260808 10:44:33.087824 3059 leader_election.cc:302] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59579d95f4b64d32a3213b0c05271c6c; no voters:
I20260808 10:44:33.088171 3059 leader_election.cc:288] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:33.088296 3066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:33.088650 3066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [term 1 LEADER]: Becoming Leader. State: Replica: 59579d95f4b64d32a3213b0c05271c6c, State: Running, Role: LEADER
I20260808 10:44:33.089035 3066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59579d95f4b64d32a3213b0c05271c6c" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44853 } }
I20260808 10:44:33.089618 3059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:33.090340 3066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59579d95f4b64d32a3213b0c05271c6c. Latest consensus state: current_term: 1 leader_uuid: "59579d95f4b64d32a3213b0c05271c6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59579d95f4b64d32a3213b0c05271c6c" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44853 } } }
I20260808 10:44:33.090469 3066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:33.090574 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.076355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34182 (local address 127.31.237.190:44853)
0808 10:44:33.076916 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:33.076934 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:33.076973 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:44:33.077636 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:33.077652 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:33.078000 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:33.078360 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:33.079308 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.080438 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.081304 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.081823 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.082011 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.084589 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:33.084595 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:33.084609 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:33.084638 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:33.086711 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.087181 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.087187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.087194 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.087325 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.087718 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.087722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.087724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.088276 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:33.088501 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:33.089162 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 10:44:33.089486 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":146,"threads_started":1}
I20260808 10:44:33.090924 3078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:33.090974 3070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59579d95f4b64d32a3213b0c05271c6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59579d95f4b64d32a3213b0c05271c6c" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44853 } } }
I20260808 10:44:33.091070 3070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:33.094504 3078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:33.100641 3078 catalog_manager.cc:1347] Generated new cluster ID: 388476622ce5493fb75d214489e5fd82
I20260808 10:44:33.100749 3078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:33.109834 3078 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:33.110903 3078 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:33.119905 3078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c: Generated new TSK 0
I20260808 10:44:33.120775 3078 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:44:33.157335 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44853
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:33.280203 3101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:33.280516 3101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:33.280598 3101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:33.284657 3101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:33.284747 3101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:33.284825 3101 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:44:33.284869 3101 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:44:33.284940 3101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:44:33.290532 3101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.237.190:44853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:33.291733 3101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:33.292824 3101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:33.302588 3114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:33.302639 3111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:33.302774 3101 server_base.cc:1034] running on GCE node
W20260808 10:44:33.302909 3112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:33.303382 3101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:33.304047 3101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:33.305212 3101 hybrid_clock.cc:648] HybridClock initialized: now 1786185873305193 us; error 31 us; skew 500 ppm
Aug 08 10:44:33 dist-test-slave-twwt krb5kdc[2901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185873, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:33.308128 3101 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:44:33.309044 3101 webserver.cc:460] Webserver started at http://127.31.237.129:35959/ using document root <none> and password file <none>
I20260808 10:44:33.309623 3101 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:33.309684 3101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:33.309908 3101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:33.312024 3101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "b38bf494febe404db5639552d247875e"
format_stamp: "Formatted at 2026-08-08 10:44:33 on dist-test-slave-twwt"
server_key: "460e31888599c32e6a096cfde9bb0f6e"
server_key_iv: "1452fda8fabfcc8538ba8df6bc443f46"
server_key_version: "encryptionkey@0"
I20260808 10:44:33.312595 3101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "b38bf494febe404db5639552d247875e"
format_stamp: "Formatted at 2026-08-08 10:44:33 on dist-test-slave-twwt"
server_key: "460e31888599c32e6a096cfde9bb0f6e"
server_key_iv: "1452fda8fabfcc8538ba8df6bc443f46"
server_key_version: "encryptionkey@0"
I20260808 10:44:33.316416 3101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 10:44:33.318811 3127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:33.320045 3101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:44:33.320215 3101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "b38bf494febe404db5639552d247875e"
format_stamp: "Formatted at 2026-08-08 10:44:33 on dist-test-slave-twwt"
server_key: "460e31888599c32e6a096cfde9bb0f6e"
server_key_iv: "1452fda8fabfcc8538ba8df6bc443f46"
server_key_version: "encryptionkey@0"
I20260808 10:44:33.320336 3101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:33.340601 3101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:33.344219 3101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:33.344398 3101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:33.345808 3101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:33.345891 3101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:33.345964 3101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:33.345996 3101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:44:33 dist-test-slave-twwt krb5kdc[2901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185873, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:33.356743 3101 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:44105
I20260808 10:44:33.356819 3312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:44105 every 8 connection(s)
I20260808 10:44:33.357666 3101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:44:33.360927 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.346824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:37835 (local address 127.31.237.190:44853)
0808 10:44:33.346962 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:33.346966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:33.348059 (+ 1093us) server_negotiation.cc:407] Connection header received
0808 10:44:33.349250 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:33.349256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:33.349328 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:33.349430 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:33.351217 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.352099 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.353903 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.354192 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.354308 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.357395 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:33.357400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:33.357402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:33.357443 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:33.359104 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.359602 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.359605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.359606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.359681 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.359985 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.359988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.359989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.360167 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:33.360239 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:33.360693 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 10:44:33.360805 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:44:33.361752 3142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.347392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:44853 (local address 127.31.237.129:37835)
0808 10:44:33.347907 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:33.347953 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:33.348985 (+ 1032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:33.349749 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:33.349761 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:33.350277 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:33.351044 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.351061 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.352263 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.352267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.353768 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.353779 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.354354 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.354358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.354559 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.355306 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:33.355335 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:33.357216 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:33.359215 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:33.359221 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:33.359241 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:33.359502 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:33.359758 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:33.359761 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:33.359762 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:33.359895 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:33.360253 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:33.360258 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:33.360482 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 10:44:33.360630 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 10:44:33.360879 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":155,"threads_started":1}
I20260808 10:44:33.363092 3314 heartbeater.cc:347] Connected to a master server at 127.31.237.190:44853
I20260808 10:44:33.363233 3314 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:33.363794 3314 heartbeater.cc:511] Master 127.31.237.190:44853 requested a full tablet report, sending...
I20260808 10:44:33.365589 2976 ts_manager.cc:194] Registered new tserver with Master: b38bf494febe404db5639552d247875e (127.31.237.129:44105)
I20260808 10:44:33.366868 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 3101
I20260808 10:44:33.366983 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:44:33.367229 2976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:37835
I20260808 10:44:33.367290 32694 external_mini_cluster.cc:1428] Setting key 6c241ba2afb3e904402346d7c3912544
I20260808 10:44:33.373234 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.365579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:36923 (local address 127.31.237.190:44853)
0808 10:44:33.365711 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:33.365715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:33.365892 (+ 177us) server_negotiation.cc:407] Connection header received
0808 10:44:33.366008 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:33.366011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:33.366048 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:33.366105 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:33.366599 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.367261 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.368288 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.368595 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.368707 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.369962 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:33.369968 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:33.369973 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:33.370005 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:33.371550 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.372060 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.372063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.372064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.372136 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.372450 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.372452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.372453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.372612 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:33.372686 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:33.373026 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 10:44:33.373120 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":54}
I20260808 10:44:33.373410 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.365518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:44853 (local address 127.31.237.129:36923)
0808 10:44:33.365801 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:33.365816 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:33.365913 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:33.366150 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:33.366153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:33.366364 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:33.366491 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.366498 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.367441 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.367444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.368166 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.368179 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.368717 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.368722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.368819 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.369359 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:33.369374 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:33.369793 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:33.371665 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:33.371671 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:33.371676 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:33.371955 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:33.372224 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:33.372229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:33.372233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:33.372353 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:33.372709 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:33.372726 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:33.372842 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 10:44:33.372965 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 10:44:33.373135 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:44:33.398228 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44853
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:33.517678 3328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:33.518004 3328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:33.518088 3328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:33.522210 3328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:33.522320 3328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:33.522398 3328 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:44:33.522444 3328 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:44:33.522505 3328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:44:33.528018 3328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.237.190:44853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:33.529137 3328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:33.530203 3328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:33.540351 3345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:33.540494 3328 server_base.cc:1034] running on GCE node
W20260808 10:44:33.540354 3348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:33.540397 3346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:33.541232 3328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:33.542042 3328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:33.543265 3328 hybrid_clock.cc:648] HybridClock initialized: now 1786185873543237 us; error 40 us; skew 500 ppm
Aug 08 10:44:33 dist-test-slave-twwt krb5kdc[2901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185873, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:33.546304 3328 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:44:33.547364 3328 webserver.cc:460] Webserver started at http://127.31.237.130:35699/ using document root <none> and password file <none>
I20260808 10:44:33.548030 3328 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:33.548131 3328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:33.548389 3328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:33.550535 3328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "41e7a7ce4d58425ab46107b62f1788cc"
format_stamp: "Formatted at 2026-08-08 10:44:33 on dist-test-slave-twwt"
server_key: "ee304211f8b786db6fdfa2fc499a974a"
server_key_iv: "e0d611dcff5ecb209a3650dfa36de080"
server_key_version: "encryptionkey@0"
I20260808 10:44:33.551205 3328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "41e7a7ce4d58425ab46107b62f1788cc"
format_stamp: "Formatted at 2026-08-08 10:44:33 on dist-test-slave-twwt"
server_key: "ee304211f8b786db6fdfa2fc499a974a"
server_key_iv: "e0d611dcff5ecb209a3650dfa36de080"
server_key_version: "encryptionkey@0"
I20260808 10:44:33.555200 3328 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:44:33.557914 3358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:33.559737 3328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:33.559937 3328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "41e7a7ce4d58425ab46107b62f1788cc"
format_stamp: "Formatted at 2026-08-08 10:44:33 on dist-test-slave-twwt"
server_key: "ee304211f8b786db6fdfa2fc499a974a"
server_key_iv: "e0d611dcff5ecb209a3650dfa36de080"
server_key_version: "encryptionkey@0"
I20260808 10:44:33.560081 3328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:33.586457 3328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:33.590152 3328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:33.590396 3328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:33.592419 3328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:33.592502 3328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:33.592587 3328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:33.592617 3328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:44:33 dist-test-slave-twwt krb5kdc[2901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185873, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:33.604887 3328 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:42497
I20260808 10:44:33.604954 3543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:42497 every 8 connection(s)
I20260808 10:44:33.606117 3328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:44:33.607805 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.593188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:41609 (local address 127.31.237.190:44853)
0808 10:44:33.593374 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:33.593379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:33.594736 (+ 1357us) server_negotiation.cc:407] Connection header received
0808 10:44:33.595900 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:33.595906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:33.595980 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:33.596098 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:33.597716 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.598263 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.599669 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.599890 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.600011 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.603551 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:33.603556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:33.603559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:33.603593 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:33.605375 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.606028 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.606031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.606032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.606114 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.606562 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.606566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.606567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.606832 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:33.606942 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:33.607547 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 10:44:33.607665 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:44:33.608548 3373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.594064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:44853 (local address 127.31.237.130:41609)
0808 10:44:33.594538 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:33.594590 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:33.595671 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:33.596397 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:33.596407 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:33.596829 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:33.597543 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.597566 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.598398 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.598401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.599536 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.599546 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.600066 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.600071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.600320 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.601007 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:33.601023 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:33.603269 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:33.605495 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:33.605501 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:33.605516 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:33.605901 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:33.606216 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:33.606219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:33.606221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:33.606417 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:33.607001 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:33.607008 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:33.607273 (+ 265us) client_negotiation.cc:769] Sending connection context
0808 10:44:33.607434 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 10:44:33.607687 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":137,"threads_started":1}
I20260808 10:44:33.609748 3544 heartbeater.cc:347] Connected to a master server at 127.31.237.190:44853
I20260808 10:44:33.609889 3544 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:33.610466 3544 heartbeater.cc:511] Master 127.31.237.190:44853 requested a full tablet report, sending...
I20260808 10:44:33.611617 2978 ts_manager.cc:194] Registered new tserver with Master: 41e7a7ce4d58425ab46107b62f1788cc (127.31.237.130:42497)
I20260808 10:44:33.612485 2978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:41609
I20260808 10:44:33.616206 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 3328
I20260808 10:44:33.616386 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:44:33.616683 32694 external_mini_cluster.cc:1428] Setting key c41a683bd29dacf145f588d663b0bd60
I20260808 10:44:33.620733 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.612259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:50935 (local address 127.31.237.190:44853)
0808 10:44:33.612370 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:33.612373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:33.613040 (+ 667us) server_negotiation.cc:407] Connection header received
0808 10:44:33.613163 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:33.613166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:33.613212 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:33.613289 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:33.613718 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.614446 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.615432 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.615642 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.615730 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.616985 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:33.616989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:33.616992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:33.617017 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:33.618711 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.619152 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.619157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.619162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.619267 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.619552 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.619558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.619562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.619895 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:33.620007 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:33.620395 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 10:44:33.620540 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:44:33.620811 3552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.612184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:44853 (local address 127.31.237.130:50935)
0808 10:44:33.612946 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:33.612962 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:33.613068 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:33.613308 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:33.613310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:33.613501 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:33.613602 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.613608 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.614580 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.614582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.615300 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.615310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.615751 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.615754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.615835 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.616400 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:33.616412 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:33.616851 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:33.618837 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:33.618840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:33.618842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:33.619047 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:33.619372 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:33.619374 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:33.619376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:33.619458 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:33.620084 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:33.620087 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:33.620209 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 10:44:33.620312 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 10:44:33.620470 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:44:33.649405 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44853
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:33.765987 3556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:33.766281 3556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:33.766368 3556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:33.770462 3556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:33.770536 3556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:33.770578 3556 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:44:33.770592 3556 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:44:33.770699 3556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:44:33.776470 3556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.237.190:44853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:33.777726 3556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:33.778864 3556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:33.789155 3566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:33.789165 3563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:33.789165 3564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:33.789446 3556 server_base.cc:1034] running on GCE node
I20260808 10:44:33.789896 3556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:33.790680 3556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:33.791858 3556 hybrid_clock.cc:648] HybridClock initialized: now 1786185873791841 us; error 30 us; skew 500 ppm
Aug 08 10:44:33 dist-test-slave-twwt krb5kdc[2901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185873, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:33.794600 3556 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:44:33.795626 3556 webserver.cc:460] Webserver started at http://127.31.237.131:41089/ using document root <none> and password file <none>
I20260808 10:44:33.796270 3556 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:33.796358 3556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:33.796581 3556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:33.798689 3556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "9cbff110d04d4ef98180c1f06eb3c4df"
format_stamp: "Formatted at 2026-08-08 10:44:33 on dist-test-slave-twwt"
server_key: "a7185393fd85143e18c429c912c7c41a"
server_key_iv: "57de2686757877f86acfd318eb4dd248"
server_key_version: "encryptionkey@0"
I20260808 10:44:33.799286 3556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "9cbff110d04d4ef98180c1f06eb3c4df"
format_stamp: "Formatted at 2026-08-08 10:44:33 on dist-test-slave-twwt"
server_key: "a7185393fd85143e18c429c912c7c41a"
server_key_iv: "57de2686757877f86acfd318eb4dd248"
server_key_version: "encryptionkey@0"
I20260808 10:44:33.803051 3556 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:44:33.805473 3577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:33.806846 3556 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:44:33.806977 3556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "9cbff110d04d4ef98180c1f06eb3c4df"
format_stamp: "Formatted at 2026-08-08 10:44:33 on dist-test-slave-twwt"
server_key: "a7185393fd85143e18c429c912c7c41a"
server_key_iv: "57de2686757877f86acfd318eb4dd248"
server_key_version: "encryptionkey@0"
I20260808 10:44:33.807090 3556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:33.820905 3556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:33.823975 3556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:33.824182 3556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:33.825699 3556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:33.825780 3556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:33.825855 3556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:33.825907 3556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:44:33 dist-test-slave-twwt krb5kdc[2901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185873, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:33.837378 3556 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:41605
I20260808 10:44:33.837459 3750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:41605 every 8 connection(s)
I20260808 10:44:33.838622 3556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:44:33.840553 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.826569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:58529 (local address 127.31.237.190:44853)
0808 10:44:33.826728 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:33.826731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:33.827774 (+ 1043us) server_negotiation.cc:407] Connection header received
0808 10:44:33.828880 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:33.828887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:33.828961 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:33.829068 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:33.830481 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.831100 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.832691 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.832923 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.833042 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.836732 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:33.836735 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:33.836737 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:33.836762 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:33.838543 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.839139 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.839143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.839144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.839240 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.839626 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.839629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.839630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.839809 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:33.839889 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:33.840335 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 10:44:33.840432 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 10:44:33.841357 3588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.827133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:44853 (local address 127.31.237.131:58529)
0808 10:44:33.827630 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:33.827672 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:33.828593 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:33.829292 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:33.829304 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:33.829667 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:33.830252 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.830265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.831230 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.831233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.832559 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.832569 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.833094 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.833098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.833278 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.833955 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:33.833975 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:33.836563 (+ 2588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:33.838700 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:33.838708 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:33.838723 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:33.839041 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:33.839362 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:33.839365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:33.839367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:33.839506 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:33.839908 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:33.839912 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:33.840128 (+ 216us) client_negotiation.cc:769] Sending connection context
0808 10:44:33.840288 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 10:44:33.840527 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":143,"threads_started":1}
I20260808 10:44:33.842546 3752 heartbeater.cc:347] Connected to a master server at 127.31.237.190:44853
I20260808 10:44:33.842698 3752 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:33.843255 3752 heartbeater.cc:511] Master 127.31.237.190:44853 requested a full tablet report, sending...
I20260808 10:44:33.844388 2976 ts_manager.cc:194] Registered new tserver with Master: 9cbff110d04d4ef98180c1f06eb3c4df (127.31.237.131:41605)
I20260808 10:44:33.845089 2976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:58529
I20260808 10:44:33.846926 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 3556
I20260808 10:44:33.847050 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:44:33.847369 32694 external_mini_cluster.cc:1428] Setting key 8d3279b9d7af3e1432ee03e338edee30
I20260808 10:44:33.849573 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:44:33.852473 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.844950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:43705 (local address 127.31.237.190:44853)
0808 10:44:33.845081 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:33.845087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:33.845238 (+ 151us) server_negotiation.cc:407] Connection header received
0808 10:44:33.845382 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:33.845388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:33.845446 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:33.845527 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:33.845994 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.846760 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.847964 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.848268 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.848389 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.849483 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:33.849487 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:33.849489 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:33.849512 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:33.850882 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.851293 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.851297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.851299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.851383 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.851676 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.851680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.851681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.851852 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:33.851958 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:33.852218 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 10:44:33.852325 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:44:33.852486 3759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.844879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:44853 (local address 127.31.237.131:43705)
0808 10:44:33.845164 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:33.845180 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:33.845288 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:33.845560 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:33.845563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:33.845778 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:33.845879 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.845885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.846907 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.846910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.847716 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.847726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.848395 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.848398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.848492 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.849016 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:33.849027 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:33.849373 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:33.850993 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:33.850997 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:33.850999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:33.851194 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:33.851484 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:33.851487 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:33.851489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:33.851577 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:33.851958 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:33.851962 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:33.852054 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 10:44:33.852162 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 10:44:33.852301 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":105,"threads_started":1}
Aug 08 10:44:33 dist-test-slave-twwt krb5kdc[2901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:44:33 dist-test-slave-twwt krb5kdc[2901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:33.877687 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.869630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34188 (local address 127.31.237.190:44853)
0808 10:44:33.869756 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:33.869760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:33.869772 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:44:33.869812 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:33.869815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:33.869854 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:33.869924 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:33.870680 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.871187 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.871996 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.872199 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.872271 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.874714 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:33.874717 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:33.874719 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:33.874741 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:33.876146 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.876573 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.876576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.876577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.876651 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:33.876891 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:33.876894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:33.876895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:33.877075 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:33.877172 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:33.877374 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 10:44:33.877475 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:44:33.880625 2976 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:44:33.883785 2976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:44:33.898487 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.892254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:44105 (local address 127.0.0.1:57748)
0808 10:44:33.892615 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:33.892663 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:33.892803 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:33.893797 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:33.893803 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:33.893830 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:33.893979 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.893984 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.895423 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.895435 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.896876 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.896890 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.897964 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.897968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.898070 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.898106 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 10:44:33.898215 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 10:44:33.898333 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":42,"thread_start_us":115,"threads_started":1}
I20260808 10:44:33.898487 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.892414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57748 (local address 127.31.237.129:44105)
0808 10:44:33.893251 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:33.893279 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:33.893307 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:44:33.893390 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:33.893395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:33.893631 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:33.893795 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:33.894097 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.895310 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.897025 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.897852 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.897967 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.898109 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:33.898244 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 10:44:33.898338 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":95,"threads_started":1}
I20260808 10:44:33.899837 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.893399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56110 (local address 127.31.237.130:42497)
0808 10:44:33.893766 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:33.893777 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:33.894251 (+ 474us) server_negotiation.cc:407] Connection header received
0808 10:44:33.894389 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:33.894393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:33.894538 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:33.894732 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:33.894965 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.896011 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.897564 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.898289 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.898439 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.898563 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:33.899610 (+ 1047us) server_negotiation.cc:299] Negotiation successful
0808 10:44:33.899700 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":118,"threads_started":1}
I20260808 10:44:33.899272 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.893052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:42497 (local address 127.0.0.1:56110)
0808 10:44:33.894154 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:33.894164 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:33.894266 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:33.894724 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:33.894727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:33.894745 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:33.894846 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.894853 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.896530 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.896533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.897438 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.897449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.898868 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.898871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.898962 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.898986 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:44:33.899054 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:44:33.899140 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"mutex_wait_us":9,"thread_start_us":98,"threads_started":1}
I20260808 10:44:33.900204 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.892469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:41605 (local address 127.0.0.1:43418)
0808 10:44:33.894063 (+ 1594us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:33.894077 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:33.894166 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:33.895054 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:33.895057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:33.895075 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:33.895151 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.895156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.896506 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.896510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.897713 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:33.897724 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.899173 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.899176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:33.899269 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.899296 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:44:33.899378 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:44:33.899458 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1518,"mutex_wait_us":21,"thread_start_us":117,"threads_started":1}
I20260808 10:44:33.901645 3202 tablet_service.cc:1467] Processing CreateTablet for tablet 5caab63ea69745a599ae255359456756 (DEFAULT_TABLE table=test-table [id=76845a36592a40919b311e8d2cfd0aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:33.902578 3202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5caab63ea69745a599ae255359456756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:33.902335 3433 tablet_service.cc:1467] Processing CreateTablet for tablet 5caab63ea69745a599ae255359456756 (DEFAULT_TABLE table=test-table [id=76845a36592a40919b311e8d2cfd0aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:33.903198 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:33.892923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.31.237.131:41605)
0808 10:44:33.894400 (+ 1477us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:33.894411 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:33.894444 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:44:33.894517 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:33.894522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:33.894731 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:33.894906 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:33.895282 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.896395 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.897845 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:33.899052 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:33.899183 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:33.899302 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:33.902942 (+ 3640us) server_negotiation.cc:299] Negotiation successful
0808 10:44:33.903048 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1366,"thread_start_us":113,"threads_started":1}
I20260808 10:44:33.903388 3433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5caab63ea69745a599ae255359456756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:33.909336 3785 tablet_bootstrap.cc:492] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e: Bootstrap starting.
I20260808 10:44:33.909430 3653 tablet_service.cc:1467] Processing CreateTablet for tablet 5caab63ea69745a599ae255359456756 (DEFAULT_TABLE table=test-table [id=76845a36592a40919b311e8d2cfd0aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:33.910559 3653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5caab63ea69745a599ae255359456756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:33.911702 3785 tablet_bootstrap.cc:654] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:33.912467 3785 log.cc:824] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:33.914417 3785 tablet_bootstrap.cc:492] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e: No bootstrap required, opened a new log
I20260808 10:44:33.914582 3785 ts_tablet_manager.cc:1397] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:44:33.915230 3786 tablet_bootstrap.cc:492] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc: Bootstrap starting.
I20260808 10:44:33.917405 3789 tablet_bootstrap.cc:492] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df: Bootstrap starting.
I20260808 10:44:33.917570 3786 tablet_bootstrap.cc:654] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:33.918411 3786 log.cc:824] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:33.919489 3789 tablet_bootstrap.cc:654] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:33.919275 3785 raft_consensus.cc:348] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:33.919579 3785 raft_consensus.cc:374] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:33.919646 3785 raft_consensus.cc:729] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38bf494febe404db5639552d247875e, State: Initialized, Role: FOLLOWER
I20260808 10:44:33.920192 3789 log.cc:824] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:33.920126 3785 consensus_queue.cc:260] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:33.920506 3786 tablet_bootstrap.cc:492] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc: No bootstrap required, opened a new log
I20260808 10:44:33.920694 3786 ts_tablet_manager.cc:1397] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:44:33.920928 3314 heartbeater.cc:503] Master 127.31.237.190:44853 was elected leader, sending a full tablet report...
I20260808 10:44:33.921131 3785 ts_tablet_manager.cc:1428] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:44:33.922971 3789 tablet_bootstrap.cc:492] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df: No bootstrap required, opened a new log
I20260808 10:44:33.923172 3789 ts_tablet_manager.cc:1397] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:44:33.924666 3786 raft_consensus.cc:348] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:33.924960 3786 raft_consensus.cc:374] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:33.925015 3786 raft_consensus.cc:729] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e7a7ce4d58425ab46107b62f1788cc, State: Initialized, Role: FOLLOWER
I20260808 10:44:33.925463 3786 consensus_queue.cc:260] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:33.926163 3544 heartbeater.cc:503] Master 127.31.237.190:44853 was elected leader, sending a full tablet report...
I20260808 10:44:33.926471 3786 ts_tablet_manager.cc:1428] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:44:33.926733 3789 raft_consensus.cc:348] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:33.926955 3789 raft_consensus.cc:374] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:33.927014 3789 raft_consensus.cc:729] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cbff110d04d4ef98180c1f06eb3c4df, State: Initialized, Role: FOLLOWER
I20260808 10:44:33.927395 3789 consensus_queue.cc:260] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:33.928041 3752 heartbeater.cc:503] Master 127.31.237.190:44853 was elected leader, sending a full tablet report...
I20260808 10:44:33.928306 3789 ts_tablet_manager.cc:1428] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260808 10:44:34.089607 3753 tablet.cc:2369] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:44:34.107550 3545 tablet.cc:2369] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:44:34.109187 3315 tablet.cc:2369] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:44:34.237994 3792 raft_consensus.cc:482] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:44:34.238294 3792 raft_consensus.cc:504] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.239727 3792 leader_election.cc:288] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41e7a7ce4d58425ab46107b62f1788cc (127.31.237.130:42497), 9cbff110d04d4ef98180c1f06eb3c4df (127.31.237.131:41605)
I20260808 10:44:34.244374 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.240064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:59057 (local address 127.31.237.130:42497)
0808 10:44:34.240205 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:34.240209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:34.240516 (+ 307us) server_negotiation.cc:407] Connection header received
0808 10:44:34.240626 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:34.240629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:34.240693 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:34.240776 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:34.241106 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.241673 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.242972 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.243613 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.243749 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.243805 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:34.244150 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 10:44:34.244242 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:44:34.244374 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.240211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:43677 (local address 127.31.237.131:41605)
0808 10:44:34.240383 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:34.240388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:34.240630 (+ 242us) server_negotiation.cc:407] Connection header received
0808 10:44:34.240811 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:34.240814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:34.240861 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:34.240941 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:34.241277 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.241851 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.243056 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.243650 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.243761 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.243812 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:34.244150 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 10:44:34.244242 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 10:44:34.244400 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.239985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:42497 (local address 127.31.237.129:59057)
0808 10:44:34.240389 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:34.240406 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:34.240532 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:34.240808 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:34.240811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:34.240831 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:34.240980 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.240988 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.241836 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.241839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.242849 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.242860 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.243773 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.243777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.243948 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.244007 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 10:44:34.244088 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 10:44:34.244185 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":47,"thread_start_us":129,"threads_started":1}
I20260808 10:44:34.244400 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.240109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:41605 (local address 127.31.237.129:43677)
0808 10:44:34.240505 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:34.240520 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:34.240644 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:34.240974 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:34.240977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:34.240993 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:34.241126 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.241134 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.241998 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.242001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.242914 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.242944 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.243773 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.243777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.243947 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.244003 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 10:44:34.244088 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:44:34.244185 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":90,"thread_start_us":115,"threads_started":1}
I20260808 10:44:34.245376 3687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5caab63ea69745a599ae255359456756" candidate_uuid: "b38bf494febe404db5639552d247875e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" is_pre_election: true
I20260808 10:44:34.245360 3465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5caab63ea69745a599ae255359456756" candidate_uuid: "b38bf494febe404db5639552d247875e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e7a7ce4d58425ab46107b62f1788cc" is_pre_election: true
I20260808 10:44:34.245721 3465 raft_consensus.cc:2454] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38bf494febe404db5639552d247875e in term 0.
I20260808 10:44:34.245721 3687 raft_consensus.cc:2454] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38bf494febe404db5639552d247875e in term 0.
I20260808 10:44:34.246320 3131 leader_election.cc:302] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 41e7a7ce4d58425ab46107b62f1788cc, 9cbff110d04d4ef98180c1f06eb3c4df, b38bf494febe404db5639552d247875e; no voters:
I20260808 10:44:34.246680 3792 raft_consensus.cc:2781] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:44:34.246784 3792 raft_consensus.cc:482] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:44:34.246850 3792 raft_consensus.cc:3037] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:34.247987 3792 raft_consensus.cc:504] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.248377 3792 leader_election.cc:288] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [CANDIDATE]: Term 1 election: Requested vote from peers 41e7a7ce4d58425ab46107b62f1788cc (127.31.237.130:42497), 9cbff110d04d4ef98180c1f06eb3c4df (127.31.237.131:41605)
I20260808 10:44:34.248828 3687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5caab63ea69745a599ae255359456756" candidate_uuid: "b38bf494febe404db5639552d247875e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cbff110d04d4ef98180c1f06eb3c4df"
I20260808 10:44:34.248836 3465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5caab63ea69745a599ae255359456756" candidate_uuid: "b38bf494febe404db5639552d247875e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e7a7ce4d58425ab46107b62f1788cc"
I20260808 10:44:34.248984 3687 raft_consensus.cc:3037] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:34.248998 3465 raft_consensus.cc:3037] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:34.250088 3465 raft_consensus.cc:2454] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38bf494febe404db5639552d247875e in term 1.
I20260808 10:44:34.250334 3687 raft_consensus.cc:2454] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38bf494febe404db5639552d247875e in term 1.
I20260808 10:44:34.250497 3129 leader_election.cc:302] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41e7a7ce4d58425ab46107b62f1788cc, b38bf494febe404db5639552d247875e; no voters:
I20260808 10:44:34.250726 3792 raft_consensus.cc:2781] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:34.250982 3792 raft_consensus.cc:686] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [term 1 LEADER]: Becoming Leader. State: Replica: b38bf494febe404db5639552d247875e, State: Running, Role: LEADER
I20260808 10:44:34.251372 3792 consensus_queue.cc:237] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.254921 2978 catalog_manager.cc:5495] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e reported cstate change: term changed from 0 to 1, leader changed from <none> to b38bf494febe404db5639552d247875e (127.31.237.129). New cstate: current_term: 1 leader_uuid: "b38bf494febe404db5639552d247875e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } health_report { overall_health: UNKNOWN } } }
Aug 08 10:44:34 dist-test-slave-twwt krb5kdc[2901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185873, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:34.397927 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.388615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:44853 (local address 127.0.0.1:34190)
0808 10:44:34.388993 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:34.389008 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:34.389129 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:34.389482 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:34.389485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:34.390035 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:34.390165 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.390173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.391084 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.391091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.391666 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.391674 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.392232 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.392235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.392307 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.392912 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:34.392923 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:34.394488 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:34.396179 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:34.396192 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:34.396200 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:34.396452 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:34.396749 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:34.396752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:34.396754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:34.396843 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:34.397252 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:34.397262 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:34.397384 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 10:44:34.397504 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 10:44:34.397702 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":107,"threads_started":1}
I20260808 10:44:34.397927 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.388682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34190 (local address 127.31.237.190:44853)
0808 10:44:34.389009 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:34.389016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:34.389091 (+ 75us) server_negotiation.cc:407] Connection header received
0808 10:44:34.389304 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:34.389306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:34.389347 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:34.389465 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:34.390290 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.390981 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.391794 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.392116 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.392224 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.394671 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:34.394674 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:34.394676 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:34.394697 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:34.396060 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:34.396559 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:34.396564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:34.396566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:34.396650 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:34.396932 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:34.396936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:34.396938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:34.397130 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:34.397239 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:34.397563 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 10:44:34.397770 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":268,"thread_start_us":158,"threads_started":1}
I20260808 10:44:34.402477 2978 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:34190:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:44:34.403331 2978 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:44:34.408619 3202 tablet_service.cc:1467] Processing CreateTablet for tablet f6c1bf89178341b89e11d03f990f3b22 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f02d8e69ca2f4e7691bf65d7f8c12a0d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:44:34.408888 3653 tablet_service.cc:1467] Processing CreateTablet for tablet f6c1bf89178341b89e11d03f990f3b22 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f02d8e69ca2f4e7691bf65d7f8c12a0d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:44:34.408784 3433 tablet_service.cc:1467] Processing CreateTablet for tablet f6c1bf89178341b89e11d03f990f3b22 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f02d8e69ca2f4e7691bf65d7f8c12a0d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:44:34.409464 3653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6c1bf89178341b89e11d03f990f3b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:34.409492 3202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6c1bf89178341b89e11d03f990f3b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:34.409562 3433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6c1bf89178341b89e11d03f990f3b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:34.412631 3789 tablet_bootstrap.cc:492] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df: Bootstrap starting.
I20260808 10:44:34.412642 3785 tablet_bootstrap.cc:492] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e: Bootstrap starting.
I20260808 10:44:34.412638 3786 tablet_bootstrap.cc:492] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc: Bootstrap starting.
I20260808 10:44:34.413620 3785 tablet_bootstrap.cc:654] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:34.413620 3789 tablet_bootstrap.cc:654] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:34.413615 3786 tablet_bootstrap.cc:654] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:34.415139 3789 tablet_bootstrap.cc:492] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df: No bootstrap required, opened a new log
I20260808 10:44:34.415252 3789 ts_tablet_manager.cc:1397] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:44:34.415243 3785 tablet_bootstrap.cc:492] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e: No bootstrap required, opened a new log
I20260808 10:44:34.415365 3786 tablet_bootstrap.cc:492] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc: No bootstrap required, opened a new log
I20260808 10:44:34.415364 3785 ts_tablet_manager.cc:1397] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:44:34.415452 3786 ts_tablet_manager.cc:1397] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:44:34.415732 3789 raft_consensus.cc:348] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.415863 3789 raft_consensus.cc:374] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:34.415906 3789 raft_consensus.cc:729] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cbff110d04d4ef98180c1f06eb3c4df, State: Initialized, Role: FOLLOWER
I20260808 10:44:34.415846 3785 raft_consensus.cc:348] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.415890 3786 raft_consensus.cc:348] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.415973 3785 raft_consensus.cc:374] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:34.415995 3786 raft_consensus.cc:374] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:34.416023 3785 raft_consensus.cc:729] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38bf494febe404db5639552d247875e, State: Initialized, Role: FOLLOWER
I20260808 10:44:34.416034 3786 raft_consensus.cc:729] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e7a7ce4d58425ab46107b62f1788cc, State: Initialized, Role: FOLLOWER
I20260808 10:44:34.416052 3789 consensus_queue.cc:260] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.416160 3786 consensus_queue.cc:260] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.416160 3785 consensus_queue.cc:260] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.416347 3789 ts_tablet_manager.cc:1428] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:44:34.416450 3785 ts_tablet_manager.cc:1428] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:44:34.416441 3796 tablet_replica.cc:439] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } } }
I20260808 10:44:34.416602 3796 tablet_replica.cc:442] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:44:34.416653 3792 tablet_replica.cc:439] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } } }
I20260808 10:44:34.416770 3792 tablet_replica.cc:442] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:44:34.416836 3795 tablet_replica.cc:439] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } } }
I20260808 10:44:34.416966 3795 tablet_replica.cc:442] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:44:34.417835 3786 ts_tablet_manager.cc:1428] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:44:34.544889 3795 raft_consensus.cc:482] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:44:34.545045 3795 raft_consensus.cc:504] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.546277 3795 leader_election.cc:288] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38bf494febe404db5639552d247875e (127.31.237.129:44105), 9cbff110d04d4ef98180c1f06eb3c4df (127.31.237.131:41605)
I20260808 10:44:34.550420 3820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.546606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:41605 (local address 127.31.237.130:46313)
0808 10:44:34.547067 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:34.547079 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:34.547215 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:34.547487 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:34.547490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:34.547507 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:34.547635 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.547640 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.548432 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.548438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.549258 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.549267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.549976 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.549980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.550088 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.550107 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:44:34.550196 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 10:44:34.550268 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":90,"threads_started":1}
I20260808 10:44:34.550408 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.546707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:46313 (local address 127.31.237.131:41605)
0808 10:44:34.546868 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:34.546873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:34.547174 (+ 301us) server_negotiation.cc:407] Connection header received
0808 10:44:34.547326 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:34.547330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:34.547380 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:34.547473 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:34.547745 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.548325 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.549384 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.549822 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.549960 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.550018 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:34.550216 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 10:44:34.550282 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:44:34.550863 3825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.546518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:44105 (local address 127.31.237.130:56157)
0808 10:44:34.546947 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:34.546963 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:34.547168 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:34.547549 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:34.547552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:34.547569 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:34.548149 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.548160 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.548983 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.548987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.549686 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.549694 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.550374 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.550380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.550492 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.550537 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 10:44:34.550635 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 10:44:34.550700 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":108,"threads_started":1}
I20260808 10:44:34.550904 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.546623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:56157 (local address 127.31.237.129:44105)
0808 10:44:34.546998 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:34.547004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:34.547022 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:44:34.547394 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:34.547397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:34.547443 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:34.547538 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:34.548281 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.548868 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.549823 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.550247 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.550399 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.550452 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:34.550660 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 10:44:34.550754 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":61,"threads_started":1}
I20260808 10:44:34.551047 3687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6c1bf89178341b89e11d03f990f3b22" candidate_uuid: "41e7a7ce4d58425ab46107b62f1788cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" is_pre_election: true
I20260808 10:44:34.551239 3687 raft_consensus.cc:2454] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41e7a7ce4d58425ab46107b62f1788cc in term 0.
I20260808 10:44:34.551599 3235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6c1bf89178341b89e11d03f990f3b22" candidate_uuid: "41e7a7ce4d58425ab46107b62f1788cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38bf494febe404db5639552d247875e" is_pre_election: true
I20260808 10:44:34.551625 3361 leader_election.cc:302] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41e7a7ce4d58425ab46107b62f1788cc, 9cbff110d04d4ef98180c1f06eb3c4df; no voters:
I20260808 10:44:34.551832 3235 raft_consensus.cc:2454] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41e7a7ce4d58425ab46107b62f1788cc in term 0.
I20260808 10:44:34.551932 3795 raft_consensus.cc:2781] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:44:34.551990 3795 raft_consensus.cc:482] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:44:34.552030 3795 raft_consensus.cc:3037] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:34.553064 3795 raft_consensus.cc:504] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.553393 3795 leader_election.cc:288] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [CANDIDATE]: Term 1 election: Requested vote from peers b38bf494febe404db5639552d247875e (127.31.237.129:44105), 9cbff110d04d4ef98180c1f06eb3c4df (127.31.237.131:41605)
I20260808 10:44:34.553769 3235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6c1bf89178341b89e11d03f990f3b22" candidate_uuid: "41e7a7ce4d58425ab46107b62f1788cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38bf494febe404db5639552d247875e"
I20260808 10:44:34.553787 3687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6c1bf89178341b89e11d03f990f3b22" candidate_uuid: "41e7a7ce4d58425ab46107b62f1788cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cbff110d04d4ef98180c1f06eb3c4df"
I20260808 10:44:34.553894 3235 raft_consensus.cc:3037] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:34.553908 3687 raft_consensus.cc:3037] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:34.554934 3687 raft_consensus.cc:2454] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41e7a7ce4d58425ab46107b62f1788cc in term 1.
I20260808 10:44:34.554935 3235 raft_consensus.cc:2454] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41e7a7ce4d58425ab46107b62f1788cc in term 1.
I20260808 10:44:34.555238 3360 leader_election.cc:302] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41e7a7ce4d58425ab46107b62f1788cc, b38bf494febe404db5639552d247875e; no voters:
I20260808 10:44:34.555416 3795 raft_consensus.cc:2781] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:34.555621 3795 raft_consensus.cc:686] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 LEADER]: Becoming Leader. State: Replica: 41e7a7ce4d58425ab46107b62f1788cc, State: Running, Role: LEADER
I20260808 10:44:34.555994 3795 consensus_queue.cc:237] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.558462 3826 tablet_replica.cc:439] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc: TxnStatusTablet state changed. Reason: New leader 41e7a7ce4d58425ab46107b62f1788cc. Latest consensus state: current_term: 1 leader_uuid: "41e7a7ce4d58425ab46107b62f1788cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } } }
I20260808 10:44:34.558583 3826 tablet_replica.cc:442] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:44:34.559221 3828 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:44:34.559444 3828 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 10:44:34.559433 2972 catalog_manager.cc:5495] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 41e7a7ce4d58425ab46107b62f1788cc (127.31.237.130). New cstate: current_term: 1 leader_uuid: "41e7a7ce4d58425ab46107b62f1788cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } health_report { overall_health: UNKNOWN } } }
I20260808 10:44:34.607657 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.603149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56126 (local address 127.31.237.130:42497)
0808 10:44:34.603271 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:34.603275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:34.603290 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:44:34.603450 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:34.603455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:34.603526 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:34.603631 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:34.603872 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.604642 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.605913 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.606255 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.606359 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.606734 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:34.607238 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:34.607351 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:34.607456 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 10:44:34.607512 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:44:34.607661 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.603007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:42497 (local address 127.0.0.1:56126)
0808 10:44:34.603165 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:34.603184 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:34.603286 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:34.603663 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:34.603665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:34.603672 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:44:34.603765 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.603769 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.604813 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.604817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.605774 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.605785 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.606378 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.606382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.606477 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.606534 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:44:34.607343 (+ 809us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:44:34.607358 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 10:44:34.607432 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 10:44:34.607519 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 10:44:34.613682 3235 raft_consensus.cc:1261] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e [term 1 FOLLOWER]: Refusing update from remote peer 41e7a7ce4d58425ab46107b62f1788cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:34.613734 3687 raft_consensus.cc:1261] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Refusing update from remote peer 41e7a7ce4d58425ab46107b62f1788cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:34.614379 3795 consensus_queue.cc:1035] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:34.614857 3826 consensus_queue.cc:1035] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:34.617637 3792 tablet_replica.cc:439] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e: TxnStatusTablet state changed. Reason: New leader 41e7a7ce4d58425ab46107b62f1788cc. Latest consensus state: current_term: 1 leader_uuid: "41e7a7ce4d58425ab46107b62f1788cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } } }
I20260808 10:44:34.617745 3792 tablet_replica.cc:442] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:44:34.618641 3796 tablet_replica.cc:439] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df: TxnStatusTablet state changed. Reason: New leader 41e7a7ce4d58425ab46107b62f1788cc. Latest consensus state: current_term: 1 leader_uuid: "41e7a7ce4d58425ab46107b62f1788cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } } }
I20260808 10:44:34.618753 3796 tablet_replica.cc:442] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:44:34.620805 3826 tablet_replica.cc:439] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41e7a7ce4d58425ab46107b62f1788cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } } }
I20260808 10:44:34.620925 3826 tablet_replica.cc:442] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:44:34.622262 3795 tablet_replica.cc:439] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41e7a7ce4d58425ab46107b62f1788cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } } }
I20260808 10:44:34.622368 3795 tablet_replica.cc:442] T f6c1bf89178341b89e11d03f990f3b22 P 41e7a7ce4d58425ab46107b62f1788cc: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:44:34.622922 3792 tablet_replica.cc:439] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41e7a7ce4d58425ab46107b62f1788cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } } }
I20260808 10:44:34.623050 3792 tablet_replica.cc:442] T f6c1bf89178341b89e11d03f990f3b22 P b38bf494febe404db5639552d247875e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:44:34.623490 3844 mvcc.cc:205] Tried to move back new op lower bound from 7316217342410715136 to 7316217342184120320. Current Snapshot: MvccSnapshot[applied={T|T < 7316217342410715136}]
I20260808 10:44:34.623757 3796 tablet_replica.cc:439] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41e7a7ce4d58425ab46107b62f1788cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } } }
I20260808 10:44:34.623864 3796 tablet_replica.cc:442] T f6c1bf89178341b89e11d03f990f3b22 P 9cbff110d04d4ef98180c1f06eb3c4df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:44:34.624897 3841 mvcc.cc:205] Tried to move back new op lower bound from 7316217342410715136 to 7316217342184120320. Current Snapshot: MvccSnapshot[applied={T|T < 7316217342410715136 or (T in {7316217342410715136})}]
I20260808 10:44:34.625088 3834 mvcc.cc:205] Tried to move back new op lower bound from 7316217342410715136 to 7316217342184120320. Current Snapshot: MvccSnapshot[applied={T|T < 7316217342410715136}]
I20260808 10:44:34.634238 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.630102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57754 (local address 127.31.237.129:44105)
0808 10:44:34.630226 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:34.630230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:34.630431 (+ 201us) server_negotiation.cc:407] Connection header received
0808 10:44:34.630576 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:34.630578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:34.630677 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:34.630763 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:34.631032 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.631521 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.632751 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.632968 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.633042 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.633346 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:34.633747 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:34.633834 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:34.633956 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 10:44:34.634074 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:44:34.645186 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.641465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:59301 (local address 127.31.237.130:42497)
0808 10:44:34.641591 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:34.641594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:34.641827 (+ 233us) server_negotiation.cc:407] Connection header received
0808 10:44:34.642006 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:34.642009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:34.642064 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:34.642140 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:34.642459 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.643087 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.644073 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.644341 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.644462 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.644749 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:34.644826 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:34.644904 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:34.645000 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 10:44:34.645048 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:44:34.645200 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.641366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:42497 (local address 127.31.237.129:59301)
0808 10:44:34.641736 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:34.641748 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:34.641877 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:34.642160 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:34.642163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:34.642171 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:44:34.642336 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.642343 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.643193 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.643198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.643956 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.643966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.644454 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.644458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.644553 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.644593 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:44:34.644911 (+ 318us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:44:34.644920 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:44:34.644991 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:44:34.645061 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":100,"threads_started":1}
I20260808 10:44:34.651882 3687 raft_consensus.cc:1261] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Refusing update from remote peer b38bf494febe404db5639552d247875e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:34.652031 3465 raft_consensus.cc:1261] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Refusing update from remote peer b38bf494febe404db5639552d247875e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:34.652393 3801 consensus_queue.cc:1035] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:34.652997 3801 consensus_queue.cc:1035] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [LEADER]: Connected to new peer: Peer: permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:34.678676 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.671720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:32811 (local address 127.31.237.129:44105)
0808 10:44:34.671889 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:34.671893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:34.674981 (+ 3088us) server_negotiation.cc:407] Connection header received
0808 10:44:34.675090 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:34.675092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:34.675140 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:34.675205 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:34.675516 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.676191 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.677562 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.677786 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.677894 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.678195 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:34.678273 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:34.678355 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:34.678454 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 10:44:34.678505 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:44:34.678707 3864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.671643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:44105 (local address 127.31.237.130:32811)
0808 10:44:34.674862 (+ 3219us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:34.674879 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:34.674990 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:34.675242 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:34.675247 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:34.675255 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:44:34.675402 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.675410 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.676443 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.676446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.677444 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.677454 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.677898 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.677901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.677999 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.678056 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:44:34.678364 (+ 308us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:44:34.678373 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:44:34.678443 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 10:44:34.678522 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3079,"thread_start_us":107,"threads_started":1}
I20260808 10:44:34.707329 2978 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:34.707549 2978 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:34.710513 2978 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c: Sending DeleteTablet for 3 replicas of tablet 5caab63ea69745a599ae255359456756
I20260808 10:44:34.711385 3433 tablet_service.cc:1514] Processing DeleteTablet for tablet 5caab63ea69745a599ae255359456756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:34 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:56110
I20260808 10:44:34.711592 3653 tablet_service.cc:1514] Processing DeleteTablet for tablet 5caab63ea69745a599ae255359456756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:34 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:43418
I20260808 10:44:34.711746 3202 tablet_service.cc:1514] Processing DeleteTablet for tablet 5caab63ea69745a599ae255359456756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:34 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:57748
I20260808 10:44:34.712157 3869 tablet_replica.cc:330] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df: stopping tablet replica
I20260808 10:44:34.712404 2978 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:44:34.712769 3869 raft_consensus.cc:2229] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 10:44:34.712821 2978 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:44:34.713119 3869 raft_consensus.cc:2258] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:34.714686 3869 ts_tablet_manager.cc:1905] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:34.717752 3433 tablet_service.cc:1467] Processing CreateTablet for tablet 9828f40b571347788c6ecebd78eccd9b (DEFAULT_TABLE table=test-table [id=ead029bed7a8443fa4a6741617ee629d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:34.717983 3869 ts_tablet_manager.cc:1918] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:44:34.718053 3433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9828f40b571347788c6ecebd78eccd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:34.718166 3869 log.cc:1196] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/5caab63ea69745a599ae255359456756
I20260808 10:44:34.718565 3869 ts_tablet_manager.cc:1939] T 5caab63ea69745a599ae255359456756 P 9cbff110d04d4ef98180c1f06eb3c4df: Deleting consensus metadata
I20260808 10:44:34.720474 3786 tablet_bootstrap.cc:492] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc: Bootstrap starting.
I20260808 10:44:34.720628 2953 catalog_manager.cc:4841] TS 9cbff110d04d4ef98180c1f06eb3c4df (127.31.237.131:41605): tablet 5caab63ea69745a599ae255359456756 (table test-table [id=76845a36592a40919b311e8d2cfd0aa3]) successfully deleted
I20260808 10:44:34.721122 3653 tablet_service.cc:1467] Processing CreateTablet for tablet 9828f40b571347788c6ecebd78eccd9b (DEFAULT_TABLE table=test-table [id=ead029bed7a8443fa4a6741617ee629d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:34.721361 3653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9828f40b571347788c6ecebd78eccd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:34.721377 3202 tablet_service.cc:1467] Processing CreateTablet for tablet 9828f40b571347788c6ecebd78eccd9b (DEFAULT_TABLE table=test-table [id=ead029bed7a8443fa4a6741617ee629d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:34.721632 3202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9828f40b571347788c6ecebd78eccd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:34.721792 3786 tablet_bootstrap.cc:654] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:34.722785 3870 tablet_replica.cc:330] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e: stopping tablet replica
I20260808 10:44:34.723085 3870 raft_consensus.cc:2229] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:44:34.723488 3870 raft_consensus.cc:2258] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:34.723661 3789 tablet_bootstrap.cc:492] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df: Bootstrap starting.
I20260808 10:44:34.724464 3785 tablet_bootstrap.cc:492] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e: Bootstrap starting.
I20260808 10:44:34.724656 3789 tablet_bootstrap.cc:654] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:34.725423 3785 tablet_bootstrap.cc:654] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:34.725854 3786 tablet_bootstrap.cc:492] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc: No bootstrap required, opened a new log
I20260808 10:44:34.725939 3786 ts_tablet_manager.cc:1397] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 10:44:34.726341 3786 raft_consensus.cc:348] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.726447 3786 raft_consensus.cc:374] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:34.726480 3786 raft_consensus.cc:729] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e7a7ce4d58425ab46107b62f1788cc, State: Initialized, Role: FOLLOWER
I20260808 10:44:34.726599 3786 consensus_queue.cc:260] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.727764 3786 ts_tablet_manager.cc:1428] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:44:34.728876 3870 ts_tablet_manager.cc:1905] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:34.731655 3785 tablet_bootstrap.cc:492] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e: No bootstrap required, opened a new log
I20260808 10:44:34.731757 3785 ts_tablet_manager.cc:1397] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260808 10:44:34.731897 3870 ts_tablet_manager.cc:1918] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:44:34.732043 3870 log.cc:1196] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/wal/wals/5caab63ea69745a599ae255359456756
I20260808 10:44:34.732208 3785 raft_consensus.cc:348] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.732344 3785 raft_consensus.cc:374] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:34.732393 3785 raft_consensus.cc:729] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38bf494febe404db5639552d247875e, State: Initialized, Role: FOLLOWER
I20260808 10:44:34.732538 3870 ts_tablet_manager.cc:1939] T 5caab63ea69745a599ae255359456756 P b38bf494febe404db5639552d247875e: Deleting consensus metadata
I20260808 10:44:34.732532 3785 consensus_queue.cc:260] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.732858 3785 ts_tablet_manager.cc:1428] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:44:34.734491 2951 catalog_manager.cc:4841] TS b38bf494febe404db5639552d247875e (127.31.237.129:44105): tablet 5caab63ea69745a599ae255359456756 (table test-table [id=76845a36592a40919b311e8d2cfd0aa3]) successfully deleted
I20260808 10:44:34.735242 3789 tablet_bootstrap.cc:492] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df: No bootstrap required, opened a new log
I20260808 10:44:34.735330 3789 ts_tablet_manager.cc:1397] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260808 10:44:34.735752 3789 raft_consensus.cc:348] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.735857 3789 raft_consensus.cc:374] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:34.735885 3789 raft_consensus.cc:729] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cbff110d04d4ef98180c1f06eb3c4df, State: Initialized, Role: FOLLOWER
I20260808 10:44:34.736011 3789 consensus_queue.cc:260] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.736253 3789 ts_tablet_manager.cc:1428] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:44:34.737334 3868 tablet_replica.cc:330] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc: stopping tablet replica
I20260808 10:44:34.737639 3868 raft_consensus.cc:2229] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:34.737782 3868 raft_consensus.cc:2258] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:34.739537 3868 ts_tablet_manager.cc:1905] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:34.742944 3868 ts_tablet_manager.cc:1918] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:44:34.743037 3868 log.cc:1196] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/5caab63ea69745a599ae255359456756
I20260808 10:44:34.743355 3868 ts_tablet_manager.cc:1939] T 5caab63ea69745a599ae255359456756 P 41e7a7ce4d58425ab46107b62f1788cc: Deleting consensus metadata
I20260808 10:44:34.744587 2951 catalog_manager.cc:4841] TS 41e7a7ce4d58425ab46107b62f1788cc (127.31.237.130:42497): tablet 5caab63ea69745a599ae255359456756 (table test-table [id=76845a36592a40919b311e8d2cfd0aa3]) successfully deleted
I20260808 10:44:34.883072 3796 raft_consensus.cc:482] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:44:34.883222 3796 raft_consensus.cc:504] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.884465 3796 leader_election.cc:288] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b38bf494febe404db5639552d247875e (127.31.237.129:44105), 41e7a7ce4d58425ab46107b62f1788cc (127.31.237.130:42497)
I20260808 10:44:34.889125 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.885203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:56871 (local address 127.31.237.129:44105)
0808 10:44:34.885326 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:34.885333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:34.885349 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:44:34.885406 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:34.885410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:34.885495 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:34.885583 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:34.885868 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.886558 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.888103 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.888550 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.888641 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.888682 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:34.888908 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 10:44:34.888981 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:44:34.889158 3883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.884680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:44105 (local address 127.31.237.131:56871)
0808 10:44:34.885112 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:34.885129 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:34.885234 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:34.885613 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:34.885616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:34.885643 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:34.885734 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.885739 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.886716 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.886719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.887973 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.887984 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.888665 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.888668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.888768 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.888804 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 10:44:34.888877 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:44:34.888961 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":190,"threads_started":1}
I20260808 10:44:34.889319 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.885525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:54797 (local address 127.31.237.130:42497)
0808 10:44:34.885648 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:34.885653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:34.885915 (+ 262us) server_negotiation.cc:407] Connection header received
0808 10:44:34.886027 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:34.886029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:34.886070 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:34.886131 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:34.886395 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.887169 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.888259 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.888683 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.888842 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.888904 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:34.889094 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 10:44:34.889190 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:44:34.889497 3884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.885225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:42497 (local address 127.31.237.131:54797)
0808 10:44:34.885778 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:34.885819 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:34.885929 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:34.886171 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:34.886174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:34.886192 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:34.886283 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.886289 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.887302 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.887304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.888136 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.888146 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.888846 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.888870 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.888958 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.888981 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:44:34.889060 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 10:44:34.889120 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":196,"threads_started":1}
I20260808 10:44:34.889683 3235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9828f40b571347788c6ecebd78eccd9b" candidate_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38bf494febe404db5639552d247875e" is_pre_election: true
I20260808 10:44:34.889860 3235 raft_consensus.cc:2454] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cbff110d04d4ef98180c1f06eb3c4df in term 0.
I20260808 10:44:34.890002 3465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9828f40b571347788c6ecebd78eccd9b" candidate_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e7a7ce4d58425ab46107b62f1788cc" is_pre_election: true
I20260808 10:44:34.890180 3465 raft_consensus.cc:2454] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cbff110d04d4ef98180c1f06eb3c4df in term 0.
I20260808 10:44:34.890988 3581 leader_election.cc:302] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cbff110d04d4ef98180c1f06eb3c4df, b38bf494febe404db5639552d247875e; no voters:
I20260808 10:44:34.891300 3796 raft_consensus.cc:2781] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:44:34.891358 3796 raft_consensus.cc:482] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:44:34.891386 3796 raft_consensus.cc:3037] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:34.892390 3796 raft_consensus.cc:504] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.892736 3796 leader_election.cc:288] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [CANDIDATE]: Term 1 election: Requested vote from peers b38bf494febe404db5639552d247875e (127.31.237.129:44105), 41e7a7ce4d58425ab46107b62f1788cc (127.31.237.130:42497)
I20260808 10:44:34.893124 3235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9828f40b571347788c6ecebd78eccd9b" candidate_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38bf494febe404db5639552d247875e"
I20260808 10:44:34.893204 3465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9828f40b571347788c6ecebd78eccd9b" candidate_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e7a7ce4d58425ab46107b62f1788cc"
I20260808 10:44:34.893245 3235 raft_consensus.cc:3037] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:34.893357 3465 raft_consensus.cc:3037] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:34.894317 3235 raft_consensus.cc:2454] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cbff110d04d4ef98180c1f06eb3c4df in term 1.
I20260808 10:44:34.894340 3465 raft_consensus.cc:2454] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cbff110d04d4ef98180c1f06eb3c4df in term 1.
I20260808 10:44:34.894634 3581 leader_election.cc:302] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41e7a7ce4d58425ab46107b62f1788cc, 9cbff110d04d4ef98180c1f06eb3c4df; no voters:
I20260808 10:44:34.894810 3796 raft_consensus.cc:2781] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:34.895030 3796 raft_consensus.cc:686] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 LEADER]: Becoming Leader. State: Replica: 9cbff110d04d4ef98180c1f06eb3c4df, State: Running, Role: LEADER
I20260808 10:44:34.895282 3796 consensus_queue.cc:237] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:34.897572 2978 catalog_manager.cc:5495] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cbff110d04d4ef98180c1f06eb3c4df (127.31.237.131). New cstate: current_term: 1 leader_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } health_report { overall_health: HEALTHY } } }
I20260808 10:44:34.936026 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.932388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43420 (local address 127.31.237.131:41605)
0808 10:44:34.932509 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:34.932513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:34.932564 (+ 51us) server_negotiation.cc:407] Connection header received
0808 10:44:34.932752 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:34.932757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:34.932827 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:34.932922 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:34.933180 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.933727 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.934568 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.934840 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.934924 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.935213 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:34.935682 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:34.935771 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:34.935871 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 10:44:34.935917 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:44:34.946528 3898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.943002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:42497 (local address 127.31.237.131:37761)
0808 10:44:34.943392 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:34.943405 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:34.943533 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:34.943825 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:34.943828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:34.943836 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:44:34.943954 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.943960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.944673 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.944678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.945298 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.945306 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.945766 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.945769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.945874 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.945916 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:44:34.946221 (+ 305us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:44:34.946230 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:44:34.946292 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 10:44:34.946370 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":134,"threads_started":1}
I20260808 10:44:34.946509 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.943152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:37761 (local address 127.31.237.130:42497)
0808 10:44:34.943330 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:34.943335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:34.943481 (+ 146us) server_negotiation.cc:407] Connection header received
0808 10:44:34.943653 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:34.943656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:34.943711 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:34.943801 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:34.944063 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.944562 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.945428 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.945648 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.945757 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.946066 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:34.946134 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:34.946216 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:34.946313 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 10:44:34.946361 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 10:44:34.952600 3465 raft_consensus.cc:1261] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Refusing update from remote peer 9cbff110d04d4ef98180c1f06eb3c4df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:34.952670 3235 raft_consensus.cc:1261] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e [term 1 FOLLOWER]: Refusing update from remote peer 9cbff110d04d4ef98180c1f06eb3c4df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:34.953115 3796 consensus_queue.cc:1035] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [LEADER]: Connected to new peer: Peer: permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:34.953318 3887 consensus_queue.cc:1035] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:34.972311 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.969083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:37979 (local address 127.31.237.131:41605)
0808 10:44:34.969200 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:34.969205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:34.969234 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:44:34.969271 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:34.969273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:34.969311 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:34.969370 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:34.969615 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.970253 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.971136 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.971395 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.971513 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.971799 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:34.971883 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:34.971973 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:34.972101 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 10:44:34.972156 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:44:34.972301 3864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:34.968775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:41605 (local address 127.31.237.130:37979)
0808 10:44:34.968894 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:34.968909 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:34.968990 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:34.969384 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:34.969386 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:34.969395 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:44:34.969499 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.969505 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.970355 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.970357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.971011 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:34.971020 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:34.971538 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:34.971541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:34.971628 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:34.971668 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:44:34.971993 (+ 325us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:44:34.971999 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:44:34.972076 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 10:44:34.972162 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 10:44:34.973023 3844 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 10:44:36.535640 2972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:36.535799 2972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:36.539912 2972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c: Sending DeleteTablet for 3 replicas of tablet 9828f40b571347788c6ecebd78eccd9b
I20260808 10:44:36.540522 3202 tablet_service.cc:1514] Processing DeleteTablet for tablet 9828f40b571347788c6ecebd78eccd9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:36 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:57748
I20260808 10:44:36.540526 3433 tablet_service.cc:1514] Processing DeleteTablet for tablet 9828f40b571347788c6ecebd78eccd9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:36 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:56110
I20260808 10:44:36.540585 3653 tablet_service.cc:1514] Processing DeleteTablet for tablet 9828f40b571347788c6ecebd78eccd9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:36 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:43418
I20260808 10:44:36.540896 3940 tablet_replica.cc:330] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e: stopping tablet replica
I20260808 10:44:36.541100 3940 raft_consensus.cc:2229] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:36.541862 3940 raft_consensus.cc:2258] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:36.544584 3941 tablet_replica.cc:330] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc: stopping tablet replica
I20260808 10:44:36.544778 3941 raft_consensus.cc:2229] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:36.545009 3941 raft_consensus.cc:2258] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:36.545197 3940 ts_tablet_manager.cc:1905] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:36.545341 3943 tablet_replica.cc:330] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df: stopping tablet replica
I20260808 10:44:36.545512 3943 raft_consensus.cc:2229] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:44:36.545992 3943 raft_consensus.cc:2258] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:36.546697 3941 ts_tablet_manager.cc:1905] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:36.548759 3940 ts_tablet_manager.cc:1918] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:44:36.549067 3940 log.cc:1196] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/wal/wals/9828f40b571347788c6ecebd78eccd9b
I20260808 10:44:36.549420 3940 ts_tablet_manager.cc:1939] T 9828f40b571347788c6ecebd78eccd9b P b38bf494febe404db5639552d247875e: Deleting consensus metadata
I20260808 10:44:36.550266 2951 catalog_manager.cc:4841] TS b38bf494febe404db5639552d247875e (127.31.237.129:44105): tablet 9828f40b571347788c6ecebd78eccd9b (table test-table [id=ead029bed7a8443fa4a6741617ee629d]) successfully deleted
I20260808 10:44:36.550510 3941 ts_tablet_manager.cc:1918] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:44:36.550623 3941 log.cc:1196] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/9828f40b571347788c6ecebd78eccd9b
I20260808 10:44:36.550948 3941 ts_tablet_manager.cc:1939] T 9828f40b571347788c6ecebd78eccd9b P 41e7a7ce4d58425ab46107b62f1788cc: Deleting consensus metadata
I20260808 10:44:36.552868 3943 ts_tablet_manager.cc:1905] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:36.553107 2951 catalog_manager.cc:4841] TS 41e7a7ce4d58425ab46107b62f1788cc (127.31.237.130:42497): tablet 9828f40b571347788c6ecebd78eccd9b (table test-table [id=ead029bed7a8443fa4a6741617ee629d]) successfully deleted
I20260808 10:44:36.555953 3943 ts_tablet_manager.cc:1918] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:44:36.556067 3943 log.cc:1196] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/9828f40b571347788c6ecebd78eccd9b
I20260808 10:44:36.556407 3943 ts_tablet_manager.cc:1939] T 9828f40b571347788c6ecebd78eccd9b P 9cbff110d04d4ef98180c1f06eb3c4df: Deleting consensus metadata
I20260808 10:44:36.557219 2953 catalog_manager.cc:4841] TS 9cbff110d04d4ef98180c1f06eb3c4df (127.31.237.131:41605): tablet 9828f40b571347788c6ecebd78eccd9b (table test-table [id=ead029bed7a8443fa4a6741617ee629d]) successfully deleted
Aug 08 10:44:36 dist-test-slave-twwt krb5kdc[2901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185876, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:44:36 dist-test-slave-twwt krb5kdc[2901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185876, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:36.576570 3951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:36.567721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.31.237.190:44853)
0808 10:44:36.568144 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:36.568150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:36.568166 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:44:36.568256 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:36.568258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:36.568314 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:36.568394 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:36.569090 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:36.569650 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:36.570488 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:36.570806 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:36.570890 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:36.573505 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:36.573508 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:36.573510 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:36.573535 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:36.575015 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:36.575417 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:36.575426 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:36.575427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:36.575505 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:36.575739 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:36.575741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:36.575742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:36.575918 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:36.576022 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:36.576230 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 10:44:36.576334 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":247,"threads_started":1}
I20260808 10:44:36.579591 2972 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:44:36.580051 2972 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:44:36.584977 3202 tablet_service.cc:1467] Processing CreateTablet for tablet 9a0dce23233d4da69d921608a7ba0874 (DEFAULT_TABLE table=test-table [id=4d25e5c603284360a043fbf2d31a9756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:36.585089 3653 tablet_service.cc:1467] Processing CreateTablet for tablet 9a0dce23233d4da69d921608a7ba0874 (DEFAULT_TABLE table=test-table [id=4d25e5c603284360a043fbf2d31a9756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:36.585136 3433 tablet_service.cc:1467] Processing CreateTablet for tablet 9a0dce23233d4da69d921608a7ba0874 (DEFAULT_TABLE table=test-table [id=4d25e5c603284360a043fbf2d31a9756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:36.585336 3202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a0dce23233d4da69d921608a7ba0874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:36.585362 3653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a0dce23233d4da69d921608a7ba0874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:36.585448 3433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a0dce23233d4da69d921608a7ba0874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:36.587987 3952 tablet_bootstrap.cc:492] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df: Bootstrap starting.
I20260808 10:44:36.588356 3955 tablet_bootstrap.cc:492] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e: Bootstrap starting.
I20260808 10:44:36.588356 3956 tablet_bootstrap.cc:492] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc: Bootstrap starting.
I20260808 10:44:36.589015 3952 tablet_bootstrap.cc:654] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:36.589407 3955 tablet_bootstrap.cc:654] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:36.589660 3956 tablet_bootstrap.cc:654] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:36.590312 3952 tablet_bootstrap.cc:492] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df: No bootstrap required, opened a new log
I20260808 10:44:36.590420 3952 ts_tablet_manager.cc:1397] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:36.590981 3952 raft_consensus.cc:348] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:36.591109 3956 tablet_bootstrap.cc:492] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc: No bootstrap required, opened a new log
I20260808 10:44:36.591113 3952 raft_consensus.cc:374] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:36.591151 3952 raft_consensus.cc:729] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cbff110d04d4ef98180c1f06eb3c4df, State: Initialized, Role: FOLLOWER
I20260808 10:44:36.591192 3956 ts_tablet_manager.cc:1397] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:44:36.591297 3952 consensus_queue.cc:260] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:36.591600 3952 ts_tablet_manager.cc:1428] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:44:36.591636 3956 raft_consensus.cc:348] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:36.591747 3956 raft_consensus.cc:374] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:36.591792 3956 raft_consensus.cc:729] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e7a7ce4d58425ab46107b62f1788cc, State: Initialized, Role: FOLLOWER
I20260808 10:44:36.591938 3956 consensus_queue.cc:260] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:36.592206 3955 tablet_bootstrap.cc:492] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e: No bootstrap required, opened a new log
I20260808 10:44:36.592253 3956 ts_tablet_manager.cc:1428] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:44:36.592288 3955 ts_tablet_manager.cc:1397] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 10:44:36.592738 3955 raft_consensus.cc:348] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:36.592850 3955 raft_consensus.cc:374] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:36.592891 3955 raft_consensus.cc:729] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38bf494febe404db5639552d247875e, State: Initialized, Role: FOLLOWER
I20260808 10:44:36.593040 3955 consensus_queue.cc:260] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:36.593456 3955 ts_tablet_manager.cc:1428] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:44:36.913761 3962 raft_consensus.cc:482] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:44:36.914012 3962 raft_consensus.cc:504] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:36.914525 3962 leader_election.cc:288] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41e7a7ce4d58425ab46107b62f1788cc (127.31.237.130:42497), 9cbff110d04d4ef98180c1f06eb3c4df (127.31.237.131:41605)
I20260808 10:44:36.915069 3465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a0dce23233d4da69d921608a7ba0874" candidate_uuid: "b38bf494febe404db5639552d247875e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e7a7ce4d58425ab46107b62f1788cc" is_pre_election: true
I20260808 10:44:36.915073 3686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a0dce23233d4da69d921608a7ba0874" candidate_uuid: "b38bf494febe404db5639552d247875e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" is_pre_election: true
I20260808 10:44:36.915207 3465 raft_consensus.cc:2454] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38bf494febe404db5639552d247875e in term 0.
I20260808 10:44:36.915257 3686 raft_consensus.cc:2454] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38bf494febe404db5639552d247875e in term 0.
I20260808 10:44:36.915581 3129 leader_election.cc:302] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41e7a7ce4d58425ab46107b62f1788cc, b38bf494febe404db5639552d247875e; no voters:
I20260808 10:44:36.915774 3962 raft_consensus.cc:2781] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:44:36.915836 3962 raft_consensus.cc:482] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:44:36.915880 3962 raft_consensus.cc:3037] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:36.916949 3962 raft_consensus.cc:504] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:36.917332 3962 leader_election.cc:288] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [CANDIDATE]: Term 1 election: Requested vote from peers 41e7a7ce4d58425ab46107b62f1788cc (127.31.237.130:42497), 9cbff110d04d4ef98180c1f06eb3c4df (127.31.237.131:41605)
I20260808 10:44:36.917680 3465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a0dce23233d4da69d921608a7ba0874" candidate_uuid: "b38bf494febe404db5639552d247875e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e7a7ce4d58425ab46107b62f1788cc"
I20260808 10:44:36.917735 3686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a0dce23233d4da69d921608a7ba0874" candidate_uuid: "b38bf494febe404db5639552d247875e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cbff110d04d4ef98180c1f06eb3c4df"
I20260808 10:44:36.917829 3465 raft_consensus.cc:3037] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:36.917847 3686 raft_consensus.cc:3037] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:36.918841 3686 raft_consensus.cc:2454] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38bf494febe404db5639552d247875e in term 1.
I20260808 10:44:36.919047 3465 raft_consensus.cc:2454] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38bf494febe404db5639552d247875e in term 1.
I20260808 10:44:36.919173 3131 leader_election.cc:302] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cbff110d04d4ef98180c1f06eb3c4df, b38bf494febe404db5639552d247875e; no voters:
I20260808 10:44:36.919377 3962 raft_consensus.cc:2781] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:36.919639 3962 raft_consensus.cc:686] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [term 1 LEADER]: Becoming Leader. State: Replica: b38bf494febe404db5639552d247875e, State: Running, Role: LEADER
I20260808 10:44:36.919799 3962 consensus_queue.cc:237] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:36.921828 2972 catalog_manager.cc:5495] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e reported cstate change: term changed from 0 to 1, leader changed from <none> to b38bf494febe404db5639552d247875e (127.31.237.129). New cstate: current_term: 1 leader_uuid: "b38bf494febe404db5639552d247875e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } health_report { overall_health: UNKNOWN } } }
I20260808 10:44:36.966192 3966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:36.962073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57764 (local address 127.31.237.129:44105)
0808 10:44:36.962404 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:36.962415 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:36.962771 (+ 356us) server_negotiation.cc:407] Connection header received
0808 10:44:36.962850 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:36.962853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:36.962928 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:36.963010 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:36.963349 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:36.964042 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:36.965028 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:36.965289 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:36.965458 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:36.965705 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:36.965793 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:36.965876 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:36.965974 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 10:44:36.966029 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":112,"threads_started":1}
I20260808 10:44:36.973995 3686 raft_consensus.cc:1261] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Refusing update from remote peer b38bf494febe404db5639552d247875e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:36.974076 3465 raft_consensus.cc:1261] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Refusing update from remote peer b38bf494febe404db5639552d247875e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:36.974503 3963 consensus_queue.cc:1035] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:36.974725 3962 consensus_queue.cc:1035] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [LEADER]: Connected to new peer: Peer: permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:37.014921 2972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:37.015064 2972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:37.017266 2972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c: Sending DeleteTablet for 3 replicas of tablet 9a0dce23233d4da69d921608a7ba0874
I20260808 10:44:37.017880 3433 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a0dce23233d4da69d921608a7ba0874 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:37 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:56110
I20260808 10:44:37.017985 3653 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a0dce23233d4da69d921608a7ba0874 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:37 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:43418
I20260808 10:44:37.018090 3941 tablet_replica.cc:330] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc: stopping tablet replica
I20260808 10:44:37.018208 3943 tablet_replica.cc:330] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df: stopping tablet replica
I20260808 10:44:37.018213 3941 raft_consensus.cc:2229] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:37.018325 3941 raft_consensus.cc:2258] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:37.018326 3943 raft_consensus.cc:2229] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:37.018431 3943 raft_consensus.cc:2258] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:37.018698 2972 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:44:37.019121 2972 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:44:37.019119 3941 ts_tablet_manager.cc:1905] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:37.019157 3202 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a0dce23233d4da69d921608a7ba0874 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:37 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:57748
I20260808 10:44:37.019418 3940 tablet_replica.cc:330] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e: stopping tablet replica
I20260808 10:44:37.019562 3940 raft_consensus.cc:2229] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:44:37.019775 3940 raft_consensus.cc:2258] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:37.020491 3940 ts_tablet_manager.cc:1905] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:37.021875 3941 ts_tablet_manager.cc:1918] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:44:37.021965 3941 log.cc:1196] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/9a0dce23233d4da69d921608a7ba0874
I20260808 10:44:37.022228 3941 ts_tablet_manager.cc:1939] T 9a0dce23233d4da69d921608a7ba0874 P 41e7a7ce4d58425ab46107b62f1788cc: Deleting consensus metadata
I20260808 10:44:37.023348 3940 ts_tablet_manager.cc:1918] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:44:37.023473 3940 log.cc:1196] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-0/wal/wals/9a0dce23233d4da69d921608a7ba0874
I20260808 10:44:37.023720 3940 ts_tablet_manager.cc:1939] T 9a0dce23233d4da69d921608a7ba0874 P b38bf494febe404db5639552d247875e: Deleting consensus metadata
I20260808 10:44:37.025574 2951 catalog_manager.cc:4841] TS 41e7a7ce4d58425ab46107b62f1788cc (127.31.237.130:42497): tablet 9a0dce23233d4da69d921608a7ba0874 (table test-table [id=4d25e5c603284360a043fbf2d31a9756]) successfully deleted
I20260808 10:44:37.025830 3433 tablet_service.cc:1467] Processing CreateTablet for tablet f1ce7fa0756c4d69af2773013c0bb3a5 (DEFAULT_TABLE table=test-table [id=30744cfaf33e40b3bf53607520118911]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:37.025811 3653 tablet_service.cc:1467] Processing CreateTablet for tablet f1ce7fa0756c4d69af2773013c0bb3a5 (DEFAULT_TABLE table=test-table [id=30744cfaf33e40b3bf53607520118911]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:37.026088 3433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1ce7fa0756c4d69af2773013c0bb3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:37.026108 3653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1ce7fa0756c4d69af2773013c0bb3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:37.026167 2951 catalog_manager.cc:4841] TS b38bf494febe404db5639552d247875e (127.31.237.129:44105): tablet 9a0dce23233d4da69d921608a7ba0874 (table test-table [id=4d25e5c603284360a043fbf2d31a9756]) successfully deleted
I20260808 10:44:37.026547 3202 tablet_service.cc:1467] Processing CreateTablet for tablet f1ce7fa0756c4d69af2773013c0bb3a5 (DEFAULT_TABLE table=test-table [id=30744cfaf33e40b3bf53607520118911]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:37.026836 3202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1ce7fa0756c4d69af2773013c0bb3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:37.028620 3952 tablet_bootstrap.cc:492] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df: Bootstrap starting.
I20260808 10:44:37.029575 3952 tablet_bootstrap.cc:654] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:37.029657 3956 tablet_bootstrap.cc:492] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc: Bootstrap starting.
I20260808 10:44:37.030977 3956 tablet_bootstrap.cc:654] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:37.031148 3952 tablet_bootstrap.cc:492] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df: No bootstrap required, opened a new log
I20260808 10:44:37.031241 3952 ts_tablet_manager.cc:1397] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:44:37.031644 3952 raft_consensus.cc:348] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:37.031788 3952 raft_consensus.cc:374] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:37.031831 3952 raft_consensus.cc:729] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cbff110d04d4ef98180c1f06eb3c4df, State: Initialized, Role: FOLLOWER
I20260808 10:44:37.031893 3955 tablet_bootstrap.cc:492] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e: Bootstrap starting.
I20260808 10:44:37.031968 3952 consensus_queue.cc:260] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:37.032279 3952 ts_tablet_manager.cc:1428] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:44:37.032516 3956 tablet_bootstrap.cc:492] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc: No bootstrap required, opened a new log
I20260808 10:44:37.032610 3956 ts_tablet_manager.cc:1397] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:44:37.032900 3955 tablet_bootstrap.cc:654] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:37.033051 3956 raft_consensus.cc:348] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:37.033179 3956 raft_consensus.cc:374] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:37.033219 3956 raft_consensus.cc:729] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e7a7ce4d58425ab46107b62f1788cc, State: Initialized, Role: FOLLOWER
I20260808 10:44:37.033376 3956 consensus_queue.cc:260] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:37.033829 3956 ts_tablet_manager.cc:1428] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:44:37.034641 3943 ts_tablet_manager.cc:1905] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:37.037763 3943 ts_tablet_manager.cc:1918] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:44:37.037851 3943 log.cc:1196] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/9a0dce23233d4da69d921608a7ba0874
I20260808 10:44:37.038143 3943 ts_tablet_manager.cc:1939] T 9a0dce23233d4da69d921608a7ba0874 P 9cbff110d04d4ef98180c1f06eb3c4df: Deleting consensus metadata
I20260808 10:44:37.039006 2953 catalog_manager.cc:4841] TS 9cbff110d04d4ef98180c1f06eb3c4df (127.31.237.131:41605): tablet 9a0dce23233d4da69d921608a7ba0874 (table test-table [id=4d25e5c603284360a043fbf2d31a9756]) successfully deleted
I20260808 10:44:37.039466 3955 tablet_bootstrap.cc:492] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e: No bootstrap required, opened a new log
I20260808 10:44:37.039533 3955 ts_tablet_manager.cc:1397] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260808 10:44:37.039878 3955 raft_consensus.cc:348] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:37.039966 3955 raft_consensus.cc:374] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:37.039987 3955 raft_consensus.cc:729] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38bf494febe404db5639552d247875e, State: Initialized, Role: FOLLOWER
I20260808 10:44:37.040122 3955 consensus_queue.cc:260] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:37.040377 3955 ts_tablet_manager.cc:1428] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:44:37.304116 3976 raft_consensus.cc:482] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:44:37.304278 3976 raft_consensus.cc:504] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:37.304770 3976 leader_election.cc:288] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41e7a7ce4d58425ab46107b62f1788cc (127.31.237.130:42497), 9cbff110d04d4ef98180c1f06eb3c4df (127.31.237.131:41605)
I20260808 10:44:37.305294 3465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ce7fa0756c4d69af2773013c0bb3a5" candidate_uuid: "b38bf494febe404db5639552d247875e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e7a7ce4d58425ab46107b62f1788cc" is_pre_election: true
I20260808 10:44:37.305307 3687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ce7fa0756c4d69af2773013c0bb3a5" candidate_uuid: "b38bf494febe404db5639552d247875e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" is_pre_election: true
I20260808 10:44:37.305486 3465 raft_consensus.cc:2454] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38bf494febe404db5639552d247875e in term 0.
I20260808 10:44:37.305493 3687 raft_consensus.cc:2454] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b38bf494febe404db5639552d247875e in term 0.
I20260808 10:44:37.305835 3131 leader_election.cc:302] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cbff110d04d4ef98180c1f06eb3c4df, b38bf494febe404db5639552d247875e; no voters:
I20260808 10:44:37.306047 3976 raft_consensus.cc:2781] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:44:37.306113 3976 raft_consensus.cc:482] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:44:37.306151 3976 raft_consensus.cc:3037] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:37.307406 3976 raft_consensus.cc:504] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:37.307830 3976 leader_election.cc:288] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [CANDIDATE]: Term 1 election: Requested vote from peers 41e7a7ce4d58425ab46107b62f1788cc (127.31.237.130:42497), 9cbff110d04d4ef98180c1f06eb3c4df (127.31.237.131:41605)
I20260808 10:44:37.308244 3687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ce7fa0756c4d69af2773013c0bb3a5" candidate_uuid: "b38bf494febe404db5639552d247875e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cbff110d04d4ef98180c1f06eb3c4df"
I20260808 10:44:37.308249 3465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1ce7fa0756c4d69af2773013c0bb3a5" candidate_uuid: "b38bf494febe404db5639552d247875e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e7a7ce4d58425ab46107b62f1788cc"
I20260808 10:44:37.308396 3687 raft_consensus.cc:3037] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:37.308408 3465 raft_consensus.cc:3037] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:37.309475 3687 raft_consensus.cc:2454] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38bf494febe404db5639552d247875e in term 1.
I20260808 10:44:37.309491 3465 raft_consensus.cc:2454] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b38bf494febe404db5639552d247875e in term 1.
I20260808 10:44:37.309816 3131 leader_election.cc:302] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cbff110d04d4ef98180c1f06eb3c4df, b38bf494febe404db5639552d247875e; no voters:
I20260808 10:44:37.310026 3976 raft_consensus.cc:2781] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:37.310150 3976 raft_consensus.cc:686] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [term 1 LEADER]: Becoming Leader. State: Replica: b38bf494febe404db5639552d247875e, State: Running, Role: LEADER
I20260808 10:44:37.310302 3976 consensus_queue.cc:237] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } }
I20260808 10:44:37.312177 2972 catalog_manager.cc:5495] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e reported cstate change: term changed from 0 to 1, leader changed from <none> to b38bf494febe404db5639552d247875e (127.31.237.129). New cstate: current_term: 1 leader_uuid: "b38bf494febe404db5639552d247875e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b38bf494febe404db5639552d247875e" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 44105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 } health_report { overall_health: UNKNOWN } } }
I20260808 10:44:37.339337 3687 raft_consensus.cc:1261] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Refusing update from remote peer b38bf494febe404db5639552d247875e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:37.339342 3465 raft_consensus.cc:1261] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Refusing update from remote peer b38bf494febe404db5639552d247875e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:37.339785 3976 consensus_queue.cc:1035] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cbff110d04d4ef98180c1f06eb3c4df" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:37.339982 3963 consensus_queue.cc:1035] T f1ce7fa0756c4d69af2773013c0bb3a5 P b38bf494febe404db5639552d247875e [LEADER]: Connected to new peer: Peer: permanent_uuid: "41e7a7ce4d58425ab46107b62f1788cc" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 42497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:44:37.354995 3915 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 10:44:38.912818 2972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:38.913031 2972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:38.915424 2972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59579d95f4b64d32a3213b0c05271c6c: Sending DeleteTablet for 3 replicas of tablet f1ce7fa0756c4d69af2773013c0bb3a5
I20260808 10:44:38.916026 3202 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ce7fa0756c4d69af2773013c0bb3a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:38 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:57748
I20260808 10:44:38.916142 3653 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ce7fa0756c4d69af2773013c0bb3a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:38 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:43418
I20260808 10:44:38.916222 3433 tablet_service.cc:1514] Processing DeleteTablet for tablet f1ce7fa0756c4d69af2773013c0bb3a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:38 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:56110
I20260808 10:44:38.916531 4024 tablet_replica.cc:330] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df: stopping tablet replica
I20260808 10:44:38.916685 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 3101
I20260808 10:44:38.916688 4024 raft_consensus.cc:2229] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:38.923128 4024 raft_consensus.cc:2258] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:38.923964 4024 ts_tablet_manager.cc:1905] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:38.927106 4024 ts_tablet_manager.cc:1918] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:44:38.927209 4024 log.cc:1196] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/f1ce7fa0756c4d69af2773013c0bb3a5
I20260808 10:44:38.927659 4024 ts_tablet_manager.cc:1939] T f1ce7fa0756c4d69af2773013c0bb3a5 P 9cbff110d04d4ef98180c1f06eb3c4df: Deleting consensus metadata
I20260808 10:44:38.928032 4025 tablet_replica.cc:330] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc: stopping tablet replica
I20260808 10:44:38.928188 4025 raft_consensus.cc:2229] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:38.928333 4025 raft_consensus.cc:2258] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:38.928864 2953 catalog_manager.cc:4841] TS 9cbff110d04d4ef98180c1f06eb3c4df (127.31.237.131:41605): tablet f1ce7fa0756c4d69af2773013c0bb3a5 (table test-table [id=30744cfaf33e40b3bf53607520118911]) successfully deleted
I20260808 10:44:38.929088 4025 ts_tablet_manager.cc:1905] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:38.932569 4025 ts_tablet_manager.cc:1918] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:44:38.932660 4025 log.cc:1196] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/f1ce7fa0756c4d69af2773013c0bb3a5
I20260808 10:44:38.932969 4025 ts_tablet_manager.cc:1939] T f1ce7fa0756c4d69af2773013c0bb3a5 P 41e7a7ce4d58425ab46107b62f1788cc: Deleting consensus metadata
I20260808 10:44:38.933804 2951 catalog_manager.cc:4841] TS 41e7a7ce4d58425ab46107b62f1788cc (127.31.237.130:42497): tablet f1ce7fa0756c4d69af2773013c0bb3a5 (table test-table [id=30744cfaf33e40b3bf53607520118911]) successfully deleted
W20260808 10:44:38.937899 2951 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:44105 (error 108)
I20260808 10:44:38.938138 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 3328
I20260808 10:44:38.943266 4027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:38.942725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:44105 (local address 127.0.0.1:57774)
0808 10:44:38.943044 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:38.943138 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:44105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":98,"threads_started":1}
W20260808 10:44:38.944522 2951 catalog_manager.cc:4584] TS b38bf494febe404db5639552d247875e (127.31.237.129:44105): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f1ce7fa0756c4d69af2773013c0bb3a5: Network error: Client connection negotiation failed: client connection to 127.31.237.129:44105: connect: Connection refused (error 111)
I20260808 10:44:38.953068 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 3556
I20260808 10:44:38.969607 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 2920
2026-08-08T10:44:38Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7494 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:39 dist-test-slave-twwt krb5kdc[4031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:39 dist-test-slave-twwt krb5kdc[4031](info): set up 2 sockets
Aug 08 10:44:39 dist-test-slave-twwt krb5kdc[4031](info): commencing operation
krb5kdc: starting...
W20260808 10:44:40.245613 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:44:40 dist-test-slave-twwt krb5kdc[4031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:44:40.373334 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:44845
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:40127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:44845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:40.485611 4052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:40.485927 4052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:40.486011 4052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:40.490020 4052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:40.490114 4052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:40.490159 4052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:40.490223 4052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:40.490260 4052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:40.495512 4052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:40127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:44845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:44845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:40.496577 4052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:40.497619 4052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:44:40.506973 4052 server_base.cc:1034] running on GCE node
W20260808 10:44:40.507095 4057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:40.507153 4064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:40.507315 4059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:40.507913 4052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:40.509227 4052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:40.510394 4052 hybrid_clock.cc:648] HybridClock initialized: now 1786185880510379 us; error 29 us; skew 500 ppm
Aug 08 10:44:40 dist-test-slave-twwt krb5kdc[4031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185880, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:40.513607 4052 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:44:40.514711 4052 webserver.cc:460] Webserver started at http://127.31.237.190:45903/ using document root <none> and password file <none>
I20260808 10:44:40.515317 4052 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:40.515384 4052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:40.515573 4052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:40.517789 4052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "97101415161f4e1581a837b325f19b5e"
format_stamp: "Formatted at 2026-08-08 10:44:40 on dist-test-slave-twwt"
server_key: "055e75e69551a4bffff2f152b12470ee"
server_key_iv: "16e3c08e1bb30dfdd433ef557aa14b03"
server_key_version: "encryptionkey@0"
I20260808 10:44:40.518352 4052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "97101415161f4e1581a837b325f19b5e"
format_stamp: "Formatted at 2026-08-08 10:44:40 on dist-test-slave-twwt"
server_key: "055e75e69551a4bffff2f152b12470ee"
server_key_iv: "16e3c08e1bb30dfdd433ef557aa14b03"
server_key_version: "encryptionkey@0"
I20260808 10:44:40.522403 4052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 10:44:40.525074 4080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:40.526314 4052 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:40.526486 4052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "97101415161f4e1581a837b325f19b5e"
format_stamp: "Formatted at 2026-08-08 10:44:40 on dist-test-slave-twwt"
server_key: "055e75e69551a4bffff2f152b12470ee"
server_key_iv: "16e3c08e1bb30dfdd433ef557aa14b03"
server_key_version: "encryptionkey@0"
I20260808 10:44:40.526604 4052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:40.550420 4052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:40.553530 4052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:40.553750 4052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:40.561460 4052 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:44845
I20260808 10:44:40.561527 4158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:44845 every 8 connection(s)
I20260808 10:44:40.562625 4052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:40.565706 4161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:40.570097 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 4052
I20260808 10:44:40.570233 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:40.570526 32694 external_mini_cluster.cc:1428] Setting key 2f745fccbf7b8e95d5d8db789b0e5ac4
I20260808 10:44:40.572474 4161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e: Bootstrap starting.
I20260808 10:44:40.574916 4161 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:40.575755 4161 log.cc:824] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:40.577765 4161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e: No bootstrap required, opened a new log
Aug 08 10:44:40 dist-test-slave-twwt krb5kdc[4031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:40.582559 4161 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97101415161f4e1581a837b325f19b5e" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44845 } }
I20260808 10:44:40.582930 4161 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:40.583015 4161 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97101415161f4e1581a837b325f19b5e, State: Initialized, Role: FOLLOWER
I20260808 10:44:40.583554 4161 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97101415161f4e1581a837b325f19b5e" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44845 } }
I20260808 10:44:40.583724 4161 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:40.583841 4161 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:40.583953 4161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:40.585093 4161 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97101415161f4e1581a837b325f19b5e" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44845 } }
I20260808 10:44:40.585491 4161 leader_election.cc:302] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97101415161f4e1581a837b325f19b5e; no voters:
I20260808 10:44:40.585866 4161 leader_election.cc:288] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:40.585987 4171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:40.586037 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:40.572101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39418 (local address 127.31.237.190:44845)
0808 10:44:40.572663 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:40.572681 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:40.572732 (+ 51us) server_negotiation.cc:407] Connection header received
0808 10:44:40.573390 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:40.573406 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:40.573761 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:40.574128 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:40.575036 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:40.576157 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:40.577055 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:40.577544 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:40.577726 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:40.580383 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:40.580390 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:40.580411 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:40.580469 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:40.582697 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:40.583223 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:40.583236 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:40.583242 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:40.583356 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:40.583656 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:40.583659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:40.583661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:40.584238 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:40.584465 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:40.584754 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 10:44:40.585068 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":163,"threads_started":1}
I20260808 10:44:40.586218 4171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [term 1 LEADER]: Becoming Leader. State: Replica: 97101415161f4e1581a837b325f19b5e, State: Running, Role: LEADER
I20260808 10:44:40.586556 4171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97101415161f4e1581a837b325f19b5e" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44845 } }
I20260808 10:44:40.587371 4161 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:40.589578 4172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97101415161f4e1581a837b325f19b5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97101415161f4e1581a837b325f19b5e" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44845 } } }
I20260808 10:44:40.589618 4173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97101415161f4e1581a837b325f19b5e. Latest consensus state: current_term: 1 leader_uuid: "97101415161f4e1581a837b325f19b5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97101415161f4e1581a837b325f19b5e" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44845 } } }
I20260808 10:44:40.589715 4172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:40.589794 4173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e [sys.catalog]: This master's current role is: LEADER
W20260808 10:44:40.592782 4190 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:40.592885 4190 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:40.592979 4189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:40.593771 4189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:40.599444 4189 catalog_manager.cc:1347] Generated new cluster ID: d6b92e7606f24518bdbefd8418ce370b
I20260808 10:44:40.599514 4189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:40.613572 4189 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:40.614967 4189 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:40.619133 4189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97101415161f4e1581a837b325f19b5e: Generated new TSK 0
I20260808 10:44:40.619830 4189 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:44:40.653708 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44845
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:40127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:40.807040 4197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:40.807312 4197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:40.807356 4197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:40.812244 4197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:40.812445 4197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:40.812848 4197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:44:40.820346 4197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:40127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:40.821550 4197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:40.822820 4197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:44:40.834190 4197 server_base.cc:1034] running on GCE node
W20260808 10:44:40.834158 4204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:40.834136 4203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:40.834129 4207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:40.834918 4197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:40.835670 4197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:40.836872 4197 hybrid_clock.cc:648] HybridClock initialized: now 1786185880836833 us; error 50 us; skew 500 ppm
Aug 08 10:44:40 dist-test-slave-twwt krb5kdc[4031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185880, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:40.840891 4197 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:44:40.843428 4197 webserver.cc:460] Webserver started at http://127.31.237.129:45631/ using document root <none> and password file <none>
I20260808 10:44:40.844151 4197 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:40.844303 4197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:40.844559 4197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:40.846837 4197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "0bf91c98e03546edb43043fb13de4863"
format_stamp: "Formatted at 2026-08-08 10:44:40 on dist-test-slave-twwt"
server_key: "4eac5779b2cd5ee4e593111bd505dd73"
server_key_iv: "7307a4c9e3e8b640323b14f6f3cdcb6a"
server_key_version: "encryptionkey@0"
I20260808 10:44:40.847721 4197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "0bf91c98e03546edb43043fb13de4863"
format_stamp: "Formatted at 2026-08-08 10:44:40 on dist-test-slave-twwt"
server_key: "4eac5779b2cd5ee4e593111bd505dd73"
server_key_iv: "7307a4c9e3e8b640323b14f6f3cdcb6a"
server_key_version: "encryptionkey@0"
I20260808 10:44:40.851800 4197 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 10:44:40.855103 4217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:40.856521 4197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:44:40.856765 4197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "0bf91c98e03546edb43043fb13de4863"
format_stamp: "Formatted at 2026-08-08 10:44:40 on dist-test-slave-twwt"
server_key: "4eac5779b2cd5ee4e593111bd505dd73"
server_key_iv: "7307a4c9e3e8b640323b14f6f3cdcb6a"
server_key_version: "encryptionkey@0"
I20260808 10:44:40.856900 4197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:40.873569 4197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:40.877300 4197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:40.877535 4197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:40.878196 4197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:40.879164 4197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:40.879240 4197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:40.879314 4197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:40.879365 4197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:40.889590 4197 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:36877
I20260808 10:44:40.889621 4392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:36877 every 8 connection(s)
I20260808 10:44:40.890551 4197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:44:40.896166 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 4197
I20260808 10:44:40.896313 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:44:40.896610 32694 external_mini_cluster.cc:1428] Setting key 64867d5398e774cecfb93b31ff2ff759
Aug 08 10:44:40 dist-test-slave-twwt krb5kdc[4031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185880, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:40.905230 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:40.892452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:59493 (local address 127.31.237.190:44845)
0808 10:44:40.892600 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:40.892604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:40.893421 (+ 817us) server_negotiation.cc:407] Connection header received
0808 10:44:40.894914 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:40.894917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:40.894980 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:40.895062 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:40.896258 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:40.896951 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:40.898320 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:40.898560 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:40.898716 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:40.901272 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:40.901278 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:40.901286 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:40.901310 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:40.902780 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:40.903528 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:40.903531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:40.903532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:40.903616 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:40.904029 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:40.904031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:40.904032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:40.904206 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:40.904284 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:40.904999 (+ 715us) server_negotiation.cc:299] Negotiation successful
0808 10:44:40.905106 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:44:40.906510 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:40.892798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:44845 (local address 127.31.237.129:59493)
0808 10:44:40.893259 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:40.893305 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:40.894600 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:40.895237 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:40.895244 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:40.895605 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:40.896097 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:40.896111 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:40.897081 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:40.897085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:40.898186 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:40.898197 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:40.898756 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:40.898760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:40.898946 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:40.899562 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:40.899579 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:40.901104 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:40.902923 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:40.902933 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:40.902952 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:40.903381 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:40.903718 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:40.903724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:40.903726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:40.903926 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:40.904325 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:40.904342 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:40.904709 (+ 367us) client_negotiation.cc:769] Sending connection context
0808 10:44:40.904951 (+ 242us) client_negotiation.cc:240] Negotiation successful
0808 10:44:40.905369 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":125,"threads_started":1}
I20260808 10:44:40.907896 4393 heartbeater.cc:347] Connected to a master server at 127.31.237.190:44845
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:44:40.908170 4393 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:40.908669 4393 heartbeater.cc:511] Master 127.31.237.190:44845 requested a full tablet report, sending...
I20260808 10:44:40.910318 4105 ts_manager.cc:194] Registered new tserver with Master: 0bf91c98e03546edb43043fb13de4863 (127.31.237.129:36877)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
I20260808 10:44:40.911697 4105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:59493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:44:40.930583 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44845
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:40127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:41.050655 4399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:41.050963 4399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:41.051041 4399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:41.055110 4399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:41.055190 4399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:41.055279 4399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:44:41.060644 4399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:40127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:41.061810 4399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:41.062917 4399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:41.072999 4416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:41.073046 4406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:41.072999 4411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:41.073170 4399 server_base.cc:1034] running on GCE node
I20260808 10:44:41.073582 4399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:41.074209 4399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:41.075387 4399 hybrid_clock.cc:648] HybridClock initialized: now 1786185881075371 us; error 26 us; skew 500 ppm
Aug 08 10:44:41 dist-test-slave-twwt krb5kdc[4031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185881, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:41.078220 4399 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:44:41.079200 4399 webserver.cc:460] Webserver started at http://127.31.237.130:45147/ using document root <none> and password file <none>
I20260808 10:44:41.079741 4399 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:41.079800 4399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:41.079967 4399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:41.081969 4399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "ccbac4a390b24dd3a69bfb66dc742a35"
format_stamp: "Formatted at 2026-08-08 10:44:41 on dist-test-slave-twwt"
server_key: "f08de629b0faef1d8a7020af0330ec28"
server_key_iv: "993431021dbf6850c37cd74933826109"
server_key_version: "encryptionkey@0"
I20260808 10:44:41.082501 4399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "ccbac4a390b24dd3a69bfb66dc742a35"
format_stamp: "Formatted at 2026-08-08 10:44:41 on dist-test-slave-twwt"
server_key: "f08de629b0faef1d8a7020af0330ec28"
server_key_iv: "993431021dbf6850c37cd74933826109"
server_key_version: "encryptionkey@0"
I20260808 10:44:41.086484 4399 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 10:44:41.089263 4428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:41.090811 4399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:41.090911 4399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "ccbac4a390b24dd3a69bfb66dc742a35"
format_stamp: "Formatted at 2026-08-08 10:44:41 on dist-test-slave-twwt"
server_key: "f08de629b0faef1d8a7020af0330ec28"
server_key_iv: "993431021dbf6850c37cd74933826109"
server_key_version: "encryptionkey@0"
I20260808 10:44:41.091007 4399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:41.102447 4399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:41.105535 4399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:41.105691 4399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:41.106192 4399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:41.107065 4399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:41.107115 4399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:41.107154 4399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:41.107170 4399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:41.116211 4399 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:41481
I20260808 10:44:41.116297 4607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:41481 every 8 connection(s)
I20260808 10:44:41.117147 4399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:44:41.117835 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 4399
I20260808 10:44:41.117976 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:44:41.118263 32694 external_mini_cluster.cc:1428] Setting key daa7cc039ad0c537a05a0a85291ac602
Aug 08 10:44:41 dist-test-slave-twwt krb5kdc[4031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185881, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:44:41.132139 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:41.119146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:37323 (local address 127.31.237.190:44845)
0808 10:44:41.119293 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:41.119303 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:41.120253 (+ 950us) server_negotiation.cc:407] Connection header received
0808 10:44:41.121727 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:41.121730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:41.121783 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:41.121880 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:41.123176 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:41.123996 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:41.125227 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:41.125610 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:41.125745 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:41.128250 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:41.128254 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:41.128259 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:41.128289 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:41.130003 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:41.130519 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:41.130521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:41.130523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:41.130598 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:41.131016 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:41.131020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:41.131022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:41.131213 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:41.131324 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:41.131852 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 10:44:41.131971 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
I20260808 10:44:41.133163 4613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:41.119656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:44845 (local address 127.31.237.130:37323)
0808 10:44:41.120111 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:41.120152 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:41.121514 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:41.122061 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:41.122068 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:41.122439 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:41.123001 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:41.123017 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:41.124138 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:41.124141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:41.125068 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:41.125082 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:41.125772 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:41.125774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:41.125919 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:41.126556 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:41.126573 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:41.128071 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:41.130142 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:41.130147 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:41.130160 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:41.130412 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:41.130740 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:41.130743 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:41.130745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:41.130909 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:41.131349 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:41.131357 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:41.131601 (+ 244us) client_negotiation.cc:769] Sending connection context
0808 10:44:41.131800 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 10:44:41.132127 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":145,"threads_started":1}
I20260808 10:44:41.134483 4610 heartbeater.cc:347] Connected to a master server at 127.31.237.190:44845
I20260808 10:44:41.134779 4610 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:41.135336 4610 heartbeater.cc:511] Master 127.31.237.190:44845 requested a full tablet report, sending...
I20260808 10:44:41.136471 4105 ts_manager.cc:194] Registered new tserver with Master: ccbac4a390b24dd3a69bfb66dc742a35 (127.31.237.130:41481)
I20260808 10:44:41.137065 4105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:37323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:44:41.149595 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44845
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:40127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:41.265780 4622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:41.266032 4622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:41.266074 4622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:41.270011 4622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:41.270063 4622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:41.270118 4622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:44:41.275818 4622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:40127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:41.276870 4622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:41.277904 4622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:41.288115 4628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:41.287982 4632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:41.288091 4622 server_base.cc:1034] running on GCE node
W20260808 10:44:41.288196 4634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:41.288749 4622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:41.289487 4622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:41.290689 4622 hybrid_clock.cc:648] HybridClock initialized: now 1786185881290654 us; error 43 us; skew 500 ppm
Aug 08 10:44:41 dist-test-slave-twwt krb5kdc[4031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185881, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:41.293618 4622 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:44:41.294691 4622 webserver.cc:460] Webserver started at http://127.31.237.131:38767/ using document root <none> and password file <none>
I20260808 10:44:41.295358 4622 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:41.295454 4622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:41.295680 4622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:41.297822 4622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "1dfa203ba5d34005ae10fa61f2b26678"
format_stamp: "Formatted at 2026-08-08 10:44:41 on dist-test-slave-twwt"
server_key: "28f4f727d38dc3a3bb062711921b3414"
server_key_iv: "a7ac411ff89a4b35c4b9d5245cb3e427"
server_key_version: "encryptionkey@0"
I20260808 10:44:41.298445 4622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "1dfa203ba5d34005ae10fa61f2b26678"
format_stamp: "Formatted at 2026-08-08 10:44:41 on dist-test-slave-twwt"
server_key: "28f4f727d38dc3a3bb062711921b3414"
server_key_iv: "a7ac411ff89a4b35c4b9d5245cb3e427"
server_key_version: "encryptionkey@0"
I20260808 10:44:41.302399 4622 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 10:44:41.305096 4642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:41.306404 4622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:44:41.306548 4622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "1dfa203ba5d34005ae10fa61f2b26678"
format_stamp: "Formatted at 2026-08-08 10:44:41 on dist-test-slave-twwt"
server_key: "28f4f727d38dc3a3bb062711921b3414"
server_key_iv: "a7ac411ff89a4b35c4b9d5245cb3e427"
server_key_version: "encryptionkey@0"
I20260808 10:44:41.306705 4622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:41.325177 4622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:41.328696 4622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:41.328914 4622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:41.329584 4622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:41.330505 4622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:41.330578 4622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:41.330665 4622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:41.330705 4622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:41.340540 4622 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:33013
I20260808 10:44:41.340566 4812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:33013 every 8 connection(s)
I20260808 10:44:41.341451 4622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:44:41.347175 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 4622
I20260808 10:44:41.347317 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:44:41.347630 32694 external_mini_cluster.cc:1428] Setting key 02dedd0df9a7e989912c0d3bb8311e3e
Aug 08 10:44:41 dist-test-slave-twwt krb5kdc[4031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185881, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:41.355523 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:41.343468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:41677 (local address 127.31.237.190:44845)
0808 10:44:41.343614 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:41.343617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:41.344442 (+ 825us) server_negotiation.cc:407] Connection header received
0808 10:44:41.345882 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:41.345886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:41.345942 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:41.346023 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:41.347286 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:41.347951 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:41.349133 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:41.349363 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:41.349477 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:41.351987 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:41.351991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:41.351993 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:41.352017 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:41.353292 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:41.353829 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:41.353835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:41.353837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:41.353920 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:41.354274 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:41.354290 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:41.354294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:41.354562 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:41.354723 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:41.355220 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 10:44:41.355355 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:44:41.356312 4816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:41.343849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:44845 (local address 127.31.237.131:41677)
0808 10:44:41.344298 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:41.344341 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:41.345624 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:41.346192 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:41.346199 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:41.346550 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:41.347124 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:41.347139 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:41.348104 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:41.348108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:41.348999 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:41.349008 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:41.349518 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:41.349522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:41.349696 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:41.350321 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:41.350338 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:41.351771 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:41.353422 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:41.353428 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:41.353443 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:41.353702 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:41.354020 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:41.354023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:41.354025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:41.354172 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:41.354721 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:41.354728 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:41.354977 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 10:44:41.355179 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 10:44:41.355500 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":111,"threads_started":1}
I20260808 10:44:41.357548 4813 heartbeater.cc:347] Connected to a master server at 127.31.237.190:44845
I20260808 10:44:41.357848 4813 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:41.358369 4813 heartbeater.cc:511] Master 127.31.237.190:44845 requested a full tablet report, sending...
I20260808 10:44:41.359552 4105 ts_manager.cc:194] Registered new tserver with Master: 1dfa203ba5d34005ae10fa61f2b26678 (127.31.237.131:33013)
I20260808 10:44:41.360263 4105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:41677
I20260808 10:44:41.361743 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:44:41.378640 4167 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:41.376950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39434 (local address 127.31.237.190:44845)
0808 10:44:41.377131 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:41.377137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:41.377318 (+ 181us) server_negotiation.cc:407] Connection header received
0808 10:44:41.377513 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:41.377516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:41.377560 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:41.377650 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:41.378455 (+ 805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39434: BlockingRecv error: recv got EOF from 127.0.0.1:39434 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:44:41.379694 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 4197
I20260808 10:44:41.405583 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 4399
I20260808 10:44:41.421626 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 4622
I20260808 10:44:41.437723 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 4052
2026-08-08T10:44:41Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2467 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:41 dist-test-slave-twwt krb5kdc[4832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:41 dist-test-slave-twwt krb5kdc[4832](info): set up 2 sockets
Aug 08 10:44:41 dist-test-slave-twwt krb5kdc[4832](info): commencing operation
krb5kdc: starting...
W20260808 10:44:42.692140 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:44:42 dist-test-slave-twwt krb5kdc[4832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:44:42.826143 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:43601
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:35317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:43601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:42.944325 4851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:42.944584 4851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:42.944623 4851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:42.948521 4851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:42.948570 4851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:42.948589 4851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:42.948606 4851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:42.948621 4851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:42.953965 4851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:35317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:43601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:43601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:42.954988 4851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:42.955968 4851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:42.964764 4867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:42.964764 4864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:42.964775 4861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:42.965114 4851 server_base.cc:1034] running on GCE node
I20260808 10:44:42.965777 4851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:42.967036 4851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:42.968210 4851 hybrid_clock.cc:648] HybridClock initialized: now 1786185882968172 us; error 50 us; skew 500 ppm
Aug 08 10:44:42 dist-test-slave-twwt krb5kdc[4832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185882, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:42.971071 4851 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:44:42.972028 4851 webserver.cc:460] Webserver started at http://127.31.237.190:43155/ using document root <none> and password file <none>
I20260808 10:44:42.972566 4851 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:42.972625 4851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:42.972810 4851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:42.975023 4851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "69bd2e98c652488499f9b29bd89c0de7"
format_stamp: "Formatted at 2026-08-08 10:44:42 on dist-test-slave-twwt"
server_key: "d8a72aa8913932cced70a6bbeea3bca3"
server_key_iv: "54dfc7d735f43b58e464123846f7aa5a"
server_key_version: "encryptionkey@0"
I20260808 10:44:42.975610 4851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "69bd2e98c652488499f9b29bd89c0de7"
format_stamp: "Formatted at 2026-08-08 10:44:42 on dist-test-slave-twwt"
server_key: "d8a72aa8913932cced70a6bbeea3bca3"
server_key_iv: "54dfc7d735f43b58e464123846f7aa5a"
server_key_version: "encryptionkey@0"
I20260808 10:44:42.979569 4851 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:44:42.982247 4881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:42.984114 4851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:42.984265 4851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "69bd2e98c652488499f9b29bd89c0de7"
format_stamp: "Formatted at 2026-08-08 10:44:42 on dist-test-slave-twwt"
server_key: "d8a72aa8913932cced70a6bbeea3bca3"
server_key_iv: "54dfc7d735f43b58e464123846f7aa5a"
server_key_version: "encryptionkey@0"
I20260808 10:44:42.984397 4851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:43.002151 4851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:43.005215 4851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:43.005453 4851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:43.013096 4851 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:43601
I20260808 10:44:43.013398 4959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:43601 every 8 connection(s)
I20260808 10:44:43.014242 4851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:43.017256 4960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:43.023273 4960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7: Bootstrap starting.
I20260808 10:44:43.024034 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 4851
I20260808 10:44:43.024179 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:43.024459 32694 external_mini_cluster.cc:1428] Setting key f28d0082bb1318e6c75a8c91c4899689
I20260808 10:44:43.026304 4960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:43.027159 4960 log.cc:824] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:43.029345 4960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7: No bootstrap required, opened a new log
Aug 08 10:44:43 dist-test-slave-twwt krb5kdc[4832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:43.033231 4960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69bd2e98c652488499f9b29bd89c0de7" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43601 } }
I20260808 10:44:43.033510 4960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:43.033578 4960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69bd2e98c652488499f9b29bd89c0de7, State: Initialized, Role: FOLLOWER
I20260808 10:44:43.034034 4960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69bd2e98c652488499f9b29bd89c0de7" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43601 } }
I20260808 10:44:43.034217 4960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:43.034301 4960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:43.034437 4960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:43.035645 4960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69bd2e98c652488499f9b29bd89c0de7" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43601 } }
I20260808 10:44:43.036024 4960 leader_election.cc:302] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69bd2e98c652488499f9b29bd89c0de7; no voters:
I20260808 10:44:43.036401 4960 leader_election.cc:288] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:43.036480 4970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:43.036737 4970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [term 1 LEADER]: Becoming Leader. State: Replica: 69bd2e98c652488499f9b29bd89c0de7, State: Running, Role: LEADER
I20260808 10:44:43.037125 4970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69bd2e98c652488499f9b29bd89c0de7" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43601 } }
I20260808 10:44:43.037912 4960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:43.038394 4970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69bd2e98c652488499f9b29bd89c0de7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69bd2e98c652488499f9b29bd89c0de7" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43601 } } }
I20260808 10:44:43.038738 4970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:43.038889 4971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69bd2e98c652488499f9b29bd89c0de7. Latest consensus state: current_term: 1 leader_uuid: "69bd2e98c652488499f9b29bd89c0de7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69bd2e98c652488499f9b29bd89c0de7" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43601 } } }
I20260808 10:44:43.039033 4971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:43.039696 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:43.025750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56170 (local address 127.31.237.190:43601)
0808 10:44:43.026203 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:43.026220 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:43.026256 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:44:43.027040 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:43.027061 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:43.027463 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:43.027816 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:43.028718 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:43.029744 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:43.030671 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:43.031159 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:43.031333 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:43.034104 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:43.034109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:43.034123 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:43.034152 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:43.036028 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:43.036529 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:43.036535 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:43.036548 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:43.036745 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:43.037094 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:43.037096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:43.037098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:43.037575 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:43.037791 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:43.038135 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 10:44:43.038680 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":138,"threads_started":1}
I20260808 10:44:43.042936 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 10:44:43.043363 4990 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:43.043452 4990 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:43.043514 4978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:43.044399 4978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:43.050698 4978 catalog_manager.cc:1347] Generated new cluster ID: fa43f20dcc3c486baa8ebc560edee530
I20260808 10:44:43.050835 4978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:43.061395 4978 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:43.062939 4978 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:43.072600 4978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69bd2e98c652488499f9b29bd89c0de7: Generated new TSK 0
I20260808 10:44:43.073374 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:43.068287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56186 (local address 127.31.237.190:43601)
0808 10:44:43.068414 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:43.068419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:43.068433 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:44:43.068582 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:43.068590 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:43.068632 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:43.068723 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:43.069276 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:43.069815 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:43.070768 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:43.071023 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:43.071150 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:43.072224 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:43.072228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:43.072231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:43.072853 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:43.072892 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:43.072911 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:43.073020 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:43.073140 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 10:44:43.073206 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":56}
I20260808 10:44:43.073447 4978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:44:43.082520 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 4851
2026-08-08T10:44:43Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1652 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:43 dist-test-slave-twwt krb5kdc[5008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:43 dist-test-slave-twwt krb5kdc[5008](info): set up 2 sockets
Aug 08 10:44:43 dist-test-slave-twwt krb5kdc[5008](info): commencing operation
krb5kdc: starting...
W20260808 10:44:44.348464 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:44:44 dist-test-slave-twwt krb5kdc[5008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:44:44.479563 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:39539
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:39539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:44.602392 5030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:44.602759 5030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:44.602860 5030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:44.606968 5030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:44.607065 5030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:44.607105 5030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:44.607143 5030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:44.607172 5030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:44.612491 5030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:39539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:39539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:44.613718 5030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:44.614797 5030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:44.623704 5036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:44.623690 5039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:44.623909 5030 server_base.cc:1034] running on GCE node
W20260808 10:44:44.623698 5035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:44.624553 5030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:44.625641 5030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:44.626803 5030 hybrid_clock.cc:648] HybridClock initialized: now 1786185884626765 us; error 52 us; skew 500 ppm
Aug 08 10:44:44 dist-test-slave-twwt krb5kdc[5008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185884, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:44.635497 5030 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:44:44.636638 5030 webserver.cc:460] Webserver started at http://127.31.237.190:34547/ using document root <none> and password file <none>
I20260808 10:44:44.637266 5030 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:44.637334 5030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:44.637584 5030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:44.639846 5030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "d9fc89713903416f87ee2a114d910ffb"
format_stamp: "Formatted at 2026-08-08 10:44:44 on dist-test-slave-twwt"
server_key: "bf65d4dbbdbeb64017198522d89b306c"
server_key_iv: "9ed8b9d096020ff4ee5825ebcc79ad61"
server_key_version: "encryptionkey@0"
I20260808 10:44:44.640465 5030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "d9fc89713903416f87ee2a114d910ffb"
format_stamp: "Formatted at 2026-08-08 10:44:44 on dist-test-slave-twwt"
server_key: "bf65d4dbbdbeb64017198522d89b306c"
server_key_iv: "9ed8b9d096020ff4ee5825ebcc79ad61"
server_key_version: "encryptionkey@0"
I20260808 10:44:44.644470 5030 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:44:44.647099 5049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:44.648437 5030 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:44.648599 5030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "d9fc89713903416f87ee2a114d910ffb"
format_stamp: "Formatted at 2026-08-08 10:44:44 on dist-test-slave-twwt"
server_key: "bf65d4dbbdbeb64017198522d89b306c"
server_key_iv: "9ed8b9d096020ff4ee5825ebcc79ad61"
server_key_version: "encryptionkey@0"
I20260808 10:44:44.648701 5030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:44.674628 5030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:44.677991 5030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:44.678220 5030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:44.685652 5030 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:39539
I20260808 10:44:44.685938 5131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:39539 every 8 connection(s)
I20260808 10:44:44.686750 5030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:44.687098 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 5030
I20260808 10:44:44.687237 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:44.687563 32694 external_mini_cluster.cc:1428] Setting key 954ffef197949c6a3d33af08f2b11a46
I20260808 10:44:44.690191 5132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:44.697034 5132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb: Bootstrap starting.
Aug 08 10:44:44 dist-test-slave-twwt krb5kdc[5008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:44.699550 5132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:44.700314 5132 log.cc:824] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:44.702325 5132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb: No bootstrap required, opened a new log
I20260808 10:44:44.703209 5136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:44.689051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49864 (local address 127.31.237.190:39539)
0808 10:44:44.689502 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:44.689523 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:44.689569 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:44:44.690471 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:44.690495 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:44.690937 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:44.691311 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:44.692004 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:44.693538 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:44.694737 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:44.695536 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:44.695717 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:44.698320 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:44.698325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:44.698339 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:44.698366 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:44.700160 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:44.700689 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:44.700694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:44.700699 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:44.700819 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:44.701138 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:44.701141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:44.701142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:44.701557 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:44.701781 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:44.702061 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 10:44:44.702364 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":118,"threads_started":1}
I20260808 10:44:44.706378 5132 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9fc89713903416f87ee2a114d910ffb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39539 } }
I20260808 10:44:44.706640 5132 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:44.706732 5132 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9fc89713903416f87ee2a114d910ffb, State: Initialized, Role: FOLLOWER
I20260808 10:44:44.707172 5132 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9fc89713903416f87ee2a114d910ffb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39539 } }
I20260808 10:44:44.707345 5132 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:44.707427 5132 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:44.707552 5132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:44.708458 5132 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9fc89713903416f87ee2a114d910ffb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39539 } }
I20260808 10:44:44.708811 5132 leader_election.cc:302] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9fc89713903416f87ee2a114d910ffb; no voters:
I20260808 10:44:44.709178 5132 leader_election.cc:288] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:44.709263 5139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:44.709561 5139 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [term 1 LEADER]: Becoming Leader. State: Replica: d9fc89713903416f87ee2a114d910ffb, State: Running, Role: LEADER
I20260808 10:44:44.709981 5139 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9fc89713903416f87ee2a114d910ffb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39539 } }
I20260808 10:44:44.710542 5132 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:44.711777 5141 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9fc89713903416f87ee2a114d910ffb. Latest consensus state: current_term: 1 leader_uuid: "d9fc89713903416f87ee2a114d910ffb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9fc89713903416f87ee2a114d910ffb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39539 } } }
I20260808 10:44:44.711916 5141 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:44.712226 5140 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9fc89713903416f87ee2a114d910ffb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9fc89713903416f87ee2a114d910ffb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39539 } } }
I20260808 10:44:44.712316 5152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:44.713336 5140 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:44.715476 5152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:44.721333 5152 catalog_manager.cc:1347] Generated new cluster ID: cf6611e9a79f4ec1831ec36f9aedd898
I20260808 10:44:44.721405 5152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:44.739023 5152 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:44.739907 5152 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:44.753274 5152 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9fc89713903416f87ee2a114d910ffb: Generated new TSK 0
I20260808 10:44:44.754022 5152 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:44:44.786154 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39539
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:44.905254 5165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:44.905527 5165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:44.905570 5165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:44.909646 5165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:44.909709 5165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:44.909861 5165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:44:44.915602 5165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:44.916718 5165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:44.917809 5165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:44.927860 5175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:44.927893 5180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:44.927909 5173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:44.928272 5165 server_base.cc:1034] running on GCE node
I20260808 10:44:44.928699 5165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:44.929400 5165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:44.930639 5165 hybrid_clock.cc:648] HybridClock initialized: now 1786185884930589 us; error 44 us; skew 500 ppm
Aug 08 10:44:44 dist-test-slave-twwt krb5kdc[5008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185884, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:44.933636 5165 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:44:44.934553 5165 webserver.cc:460] Webserver started at http://127.31.237.129:33235/ using document root <none> and password file <none>
I20260808 10:44:44.935180 5165 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:44.935276 5165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:44.935527 5165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:44.937628 5165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "5979c8c06a294d85a40d64559f2c64e9"
format_stamp: "Formatted at 2026-08-08 10:44:44 on dist-test-slave-twwt"
server_key: "5ed28c071c2d43c49f7125875f6e6cbd"
server_key_iv: "e09cfda904aa33a2f27312d41268f6e6"
server_key_version: "encryptionkey@0"
I20260808 10:44:44.938215 5165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "5979c8c06a294d85a40d64559f2c64e9"
format_stamp: "Formatted at 2026-08-08 10:44:44 on dist-test-slave-twwt"
server_key: "5ed28c071c2d43c49f7125875f6e6cbd"
server_key_iv: "e09cfda904aa33a2f27312d41268f6e6"
server_key_version: "encryptionkey@0"
I20260808 10:44:44.942020 5165 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:44:44.944618 5190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:44.945832 5165 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:44.945990 5165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "5979c8c06a294d85a40d64559f2c64e9"
format_stamp: "Formatted at 2026-08-08 10:44:44 on dist-test-slave-twwt"
server_key: "5ed28c071c2d43c49f7125875f6e6cbd"
server_key_iv: "e09cfda904aa33a2f27312d41268f6e6"
server_key_version: "encryptionkey@0"
I20260808 10:44:44.946110 5165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:44.962828 5165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:44.966135 5165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:44.966356 5165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:44.967039 5165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:44.967878 5165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:44.967959 5165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:44.968037 5165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:44.968093 5165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:44.977339 5165 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:41509
I20260808 10:44:44.977370 5362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:41509 every 8 connection(s)
I20260808 10:44:44.978274 5165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:44:44.984050 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 5165
I20260808 10:44:44.984189 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:44:44.984488 32694 external_mini_cluster.cc:1428] Setting key 74f8a62d360769eeb55b0fad75444697
Aug 08 10:44:44 dist-test-slave-twwt krb5kdc[5008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185884, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:44.993721 5136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:44.980188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:45633 (local address 127.31.237.190:39539)
0808 10:44:44.980327 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:44.980331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:44.981143 (+ 812us) server_negotiation.cc:407] Connection header received
0808 10:44:44.982638 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:44.982643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:44.982708 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:44.982824 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:44.984086 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:44.984753 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:44.986144 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:44.986386 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:44.986501 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:44.989079 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:44.989084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:44.989090 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:44.989124 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:44.991185 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:44.991776 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:44.991781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:44.991783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:44.991878 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:44.992353 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:44.992360 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:44.992364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:44.992670 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:44.992813 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:44.993322 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 10:44:44.993485 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:44:44.994407 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:44.980516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39539 (local address 127.31.237.129:45633)
0808 10:44:44.980983 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:44.981027 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:44.982374 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:44.983017 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:44.983024 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:44.983395 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:44.983929 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:44.983944 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:44.984889 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:44.984892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:44.986021 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:44.986031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:44.986544 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:44.986548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:44.986769 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:44.987395 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:44.987419 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:44.988893 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:44.991343 (+ 2450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:44.991350 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:44.991369 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:44.991663 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:44.991985 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:44.991989 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:44.991991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:44.992173 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:44.992802 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:44.992807 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:44.993054 (+ 247us) client_negotiation.cc:769] Sending connection context
0808 10:44:44.993221 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 10:44:44.993493 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":113,"threads_started":1}
I20260808 10:44:44.995988 5364 heartbeater.cc:347] Connected to a master server at 127.31.237.190:39539
I20260808 10:44:44.996277 5364 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:44:44.996915 5364 heartbeater.cc:511] Master 127.31.237.190:39539 requested a full tablet report, sending...
I20260808 10:44:44.998472 5080 ts_manager.cc:194] Registered new tserver with Master: 5979c8c06a294d85a40d64559f2c64e9 (127.31.237.129:41509)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
I20260808 10:44:44.999943 5080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:45633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:44:45.016084 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39539
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:45.134034 5375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:45.134358 5375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:45.134440 5375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:45.138540 5375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:45.138666 5375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:45.138761 5375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:44:45.144218 5375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:45.145409 5375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:45.146549 5375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:45.157253 5381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:45.157539 5375 server_base.cc:1034] running on GCE node
W20260808 10:44:45.157692 5382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:45.157716 5384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:45.158241 5375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:45.158980 5375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:45.160147 5375 hybrid_clock.cc:648] HybridClock initialized: now 1786185885160113 us; error 48 us; skew 500 ppm
Aug 08 10:44:45 dist-test-slave-twwt krb5kdc[5008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185885, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:45.162928 5375 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:44:45.163844 5375 webserver.cc:460] Webserver started at http://127.31.237.130:45029/ using document root <none> and password file <none>
I20260808 10:44:45.164391 5375 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:45.164461 5375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:45.164631 5375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:45.166711 5375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "4bb7df91da3d44ae994227e8fc309f40"
format_stamp: "Formatted at 2026-08-08 10:44:45 on dist-test-slave-twwt"
server_key: "b583f0c12f0280e36aa3825412c495ff"
server_key_iv: "4f346ed93e75eef1208fb7d48e9735c0"
server_key_version: "encryptionkey@0"
I20260808 10:44:45.167259 5375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "4bb7df91da3d44ae994227e8fc309f40"
format_stamp: "Formatted at 2026-08-08 10:44:45 on dist-test-slave-twwt"
server_key: "b583f0c12f0280e36aa3825412c495ff"
server_key_iv: "4f346ed93e75eef1208fb7d48e9735c0"
server_key_version: "encryptionkey@0"
I20260808 10:44:45.171068 5375 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20260808 10:44:45.173523 5395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:45.174921 5375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:44:45.175058 5375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "4bb7df91da3d44ae994227e8fc309f40"
format_stamp: "Formatted at 2026-08-08 10:44:45 on dist-test-slave-twwt"
server_key: "b583f0c12f0280e36aa3825412c495ff"
server_key_iv: "4f346ed93e75eef1208fb7d48e9735c0"
server_key_version: "encryptionkey@0"
I20260808 10:44:45.175181 5375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:45.195780 5375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:45.199054 5375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:45.199278 5375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:45.199939 5375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:45.200826 5375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:45.200901 5375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:45.200971 5375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:45.201014 5375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:45.209836 5375 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:33759
I20260808 10:44:45.209867 5569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:33759 every 8 connection(s)
I20260808 10:44:45.210888 5375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:44:45.213400 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 5375
I20260808 10:44:45.213553 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:44:45.213908 32694 external_mini_cluster.cc:1428] Setting key 9fa9daeb0528aac94089a87e38eebfd5
Aug 08 10:44:45 dist-test-slave-twwt krb5kdc[5008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185885, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:44:45.226467 5136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:45.212954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:43469 (local address 127.31.237.190:39539)
0808 10:44:45.213110 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:45.213117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:45.214131 (+ 1014us) server_negotiation.cc:407] Connection header received
0808 10:44:45.216036 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:45.216039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:45.216097 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:45.216170 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:45.217350 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:45.217879 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:45.219348 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:45.219582 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:45.219671 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:45.222683 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:45.222687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:45.222689 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:45.222712 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:45.224219 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:45.224767 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:45.224772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:45.224773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:45.224897 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:45.225324 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:45.225328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:45.225330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:45.225524 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:45.225638 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:45.226169 (+ 531us) server_negotiation.cc:299] Negotiation successful
0808 10:44:45.226300 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:44:45.227344 5572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:45.213386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39539 (local address 127.31.237.130:43469)
0808 10:44:45.213950 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:45.214013 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:45.215798 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:45.216350 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:45.216358 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:45.216718 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:45.217208 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:45.217221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:45.218015 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:45.218019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:45.219219 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:45.219235 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:45.219751 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:45.219758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:45.220012 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:45.220746 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:45.220773 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:45.222488 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:45.224356 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:45.224362 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:45.224379 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:45.224656 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:45.225006 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:45.225010 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:45.225012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:45.225216 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:45.225639 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:45.225649 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:45.225921 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 10:44:45.226123 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 10:44:45.226474 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":156,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
I20260808 10:44:45.228638 5570 heartbeater.cc:347] Connected to a master server at 127.31.237.190:39539
I20260808 10:44:45.228919 5570 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:45.229552 5570 heartbeater.cc:511] Master 127.31.237.190:39539 requested a full tablet report, sending...
I20260808 10:44:45.230803 5080 ts_manager.cc:194] Registered new tserver with Master: 4bb7df91da3d44ae994227e8fc309f40 (127.31.237.130:33759)
I20260808 10:44:45.231389 5080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:43469
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:44:45.245801 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39539
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:45.365764 5580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:45.366071 5580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:45.366147 5580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:45.370270 5580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:45.370378 5580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:45.370473 5580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:44:45.375952 5580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:45.377108 5580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:45.378229 5580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:45.388833 5587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:45.388875 5586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:45.388970 5580 server_base.cc:1034] running on GCE node
W20260808 10:44:45.388841 5590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:45.389569 5580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:45.390239 5580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:45.391403 5580 hybrid_clock.cc:648] HybridClock initialized: now 1786185885391387 us; error 27 us; skew 500 ppm
Aug 08 10:44:45 dist-test-slave-twwt krb5kdc[5008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185885, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:45.394351 5580 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:44:45.395323 5580 webserver.cc:460] Webserver started at http://127.31.237.131:39099/ using document root <none> and password file <none>
I20260808 10:44:45.395915 5580 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:45.396013 5580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:45.396255 5580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:45.398329 5580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "00bb95af0dc64f69a7f406bd0d9b95de"
format_stamp: "Formatted at 2026-08-08 10:44:45 on dist-test-slave-twwt"
server_key: "0f4fcb321bed7cfd9491ef3a867c7352"
server_key_iv: "8f912740152121dfcffdbffcbc67c01b"
server_key_version: "encryptionkey@0"
I20260808 10:44:45.398927 5580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "00bb95af0dc64f69a7f406bd0d9b95de"
format_stamp: "Formatted at 2026-08-08 10:44:45 on dist-test-slave-twwt"
server_key: "0f4fcb321bed7cfd9491ef3a867c7352"
server_key_iv: "8f912740152121dfcffdbffcbc67c01b"
server_key_version: "encryptionkey@0"
I20260808 10:44:45.402647 5580 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:45.405066 5603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:45.406281 5580 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:45.406419 5580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "00bb95af0dc64f69a7f406bd0d9b95de"
format_stamp: "Formatted at 2026-08-08 10:44:45 on dist-test-slave-twwt"
server_key: "0f4fcb321bed7cfd9491ef3a867c7352"
server_key_iv: "8f912740152121dfcffdbffcbc67c01b"
server_key_version: "encryptionkey@0"
I20260808 10:44:45.406543 5580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:45.427265 5580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:45.430706 5580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:45.430910 5580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:45.431524 5580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:45.432327 5580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:45.432375 5580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:45.432415 5580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:45.432454 5580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:45.441177 5580 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:46729
I20260808 10:44:45.441207 5772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:46729 every 8 connection(s)
I20260808 10:44:45.442060 5580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:44:45.443542 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 5580
I20260808 10:44:45.443674 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:44:45.444082 32694 external_mini_cluster.cc:1428] Setting key 2565e11831c756d7bebbc510ac565978
Aug 08 10:44:45 dist-test-slave-twwt krb5kdc[5008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185885, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:45.455433 5136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:45.443941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:57765 (local address 127.31.237.190:39539)
0808 10:44:45.444119 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:45.444124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:45.445052 (+ 928us) server_negotiation.cc:407] Connection header received
0808 10:44:45.446495 (+ 1443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:45.446497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:45.446543 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:45.446660 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:45.447795 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:45.448301 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:45.449404 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:45.449620 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:45.449694 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:45.452112 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:45.452117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:45.452122 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:45.452155 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:45.453635 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:45.454092 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:45.454094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:45.454096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:45.454168 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:45.454476 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:45.454478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:45.454479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:45.454714 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:45.454796 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:45.455231 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 10:44:45.455321 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:44:45.456225 5776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:45.444387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39539 (local address 127.31.237.131:57765)
0808 10:44:45.444907 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:45.444963 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:45.446282 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:45.446826 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:45.446834 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:45.447180 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:45.447673 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:45.447686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:45.448418 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:45.448420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:45.449299 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:45.449309 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:45.449743 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:45.449746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:45.449925 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:45.450449 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:45.450466 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:45.451927 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:45.453741 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:45.453746 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:45.453757 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:45.454006 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:45.454247 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:45.454249 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:45.454250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:45.454378 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:45.454805 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:45.454809 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:45.455028 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 10:44:45.455183 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 10:44:45.455429 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":142,"threads_started":1}
I20260808 10:44:45.457520 5773 heartbeater.cc:347] Connected to a master server at 127.31.237.190:39539
I20260808 10:44:45.457784 5773 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:45.458261 5773 heartbeater.cc:511] Master 127.31.237.190:39539 requested a full tablet report, sending...
I20260808 10:44:45.459455 5073 ts_manager.cc:194] Registered new tserver with Master: 00bb95af0dc64f69a7f406bd0d9b95de (127.31.237.131:46729)
I20260808 10:44:45.460116 5073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:57765
I20260808 10:44:45.469555 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:44:45 dist-test-slave-twwt krb5kdc[5008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185885, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 10:44:45 dist-test-slave-twwt krb5kdc[5008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185885, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:45.496685 5136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:45.488219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49878 (local address 127.31.237.190:39539)
0808 10:44:45.488352 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:45.488355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:45.488494 (+ 139us) server_negotiation.cc:407] Connection header received
0808 10:44:45.488612 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:45.488614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:45.488659 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:45.488735 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:45.489429 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:45.489938 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:45.490984 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:45.491192 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:45.491274 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:45.493908 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:45.493912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:45.493914 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:45.493938 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:45.495244 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:45.495626 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:45.495629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:45.495630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:45.495708 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:45.495942 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:45.495944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:45.495945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:45.496161 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:45.496263 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:45.496468 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 10:44:45.496564 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 10:44:45.497196 5073 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49878
I20260808 10:44:45.498106 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 5165
I20260808 10:44:45.510450 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 5375
I20260808 10:44:45.520054 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 5580
I20260808 10:44:45.538949 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 5030
2026-08-08T10:44:45Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2448 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:45 dist-test-slave-twwt krb5kdc[5795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:45 dist-test-slave-twwt krb5kdc[5795](info): set up 2 sockets
Aug 08 10:44:45 dist-test-slave-twwt krb5kdc[5795](info): commencing operation
krb5kdc: starting...
W20260808 10:44:46.784998 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:44:46 dist-test-slave-twwt krb5kdc[5795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:44:46.916134 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:35245
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:35245
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:47.033239 5816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:47.033556 5816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:47.033631 5816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:47.037762 5816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:47.037914 5816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:47.037957 5816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:47.038018 5816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:47.038055 5816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:47.043571 5816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:35245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:35245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:47.044735 5816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:47.045828 5816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:47.054924 5827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:47.054984 5826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:47.054934 5831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:47.055405 5816 server_base.cc:1034] running on GCE node
I20260808 10:44:47.056011 5816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:47.057147 5816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:47.058350 5816 hybrid_clock.cc:648] HybridClock initialized: now 1786185887058336 us; error 27 us; skew 500 ppm
Aug 08 10:44:47 dist-test-slave-twwt krb5kdc[5795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185887, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:47.061540 5816 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:44:47.062532 5816 webserver.cc:460] Webserver started at http://127.31.237.190:43825/ using document root <none> and password file <none>
I20260808 10:44:47.063225 5816 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:47.063325 5816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:47.063577 5816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:47.065718 5816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "4fde9f6d977d428bb06b60ed351e338a"
format_stamp: "Formatted at 2026-08-08 10:44:47 on dist-test-slave-twwt"
server_key: "4a0283d140fcc6fd45088c0f25511921"
server_key_iv: "43ddbb0999b7a10e6f492d78e0bc96f7"
server_key_version: "encryptionkey@0"
I20260808 10:44:47.066336 5816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "4fde9f6d977d428bb06b60ed351e338a"
format_stamp: "Formatted at 2026-08-08 10:44:47 on dist-test-slave-twwt"
server_key: "4a0283d140fcc6fd45088c0f25511921"
server_key_iv: "43ddbb0999b7a10e6f492d78e0bc96f7"
server_key_version: "encryptionkey@0"
I20260808 10:44:47.070312 5816 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:44:47.073151 5840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:47.074505 5816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 10:44:47.074694 5816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "4fde9f6d977d428bb06b60ed351e338a"
format_stamp: "Formatted at 2026-08-08 10:44:47 on dist-test-slave-twwt"
server_key: "4a0283d140fcc6fd45088c0f25511921"
server_key_iv: "43ddbb0999b7a10e6f492d78e0bc96f7"
server_key_version: "encryptionkey@0"
I20260808 10:44:47.074867 5816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:47.092839 5816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:47.096196 5816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:47.096431 5816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:47.104003 5816 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:35245
I20260808 10:44:47.104014 5923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:35245 every 8 connection(s)
I20260808 10:44:47.105036 5816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:47.107932 5924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:47.113574 5924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a: Bootstrap starting.
I20260808 10:44:47.113860 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 5816
I20260808 10:44:47.114015 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:47.114281 32694 external_mini_cluster.cc:1428] Setting key 6028a9fb6ad6ecd76f22a6250f7b330b
I20260808 10:44:47.116232 5924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:47.116978 5924 log.cc:824] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:47.119243 5924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a: No bootstrap required, opened a new log
I20260808 10:44:47.123759 5924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fde9f6d977d428bb06b60ed351e338a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 35245 } }
I20260808 10:44:47.123982 5924 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:47.124040 5924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fde9f6d977d428bb06b60ed351e338a, State: Initialized, Role: FOLLOWER
Aug 08 10:44:47 dist-test-slave-twwt krb5kdc[5795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:47.124508 5924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fde9f6d977d428bb06b60ed351e338a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 35245 } }
I20260808 10:44:47.124635 5924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:47.124747 5924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:47.124902 5924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:47.125756 5924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fde9f6d977d428bb06b60ed351e338a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 35245 } }
I20260808 10:44:47.126116 5924 leader_election.cc:302] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fde9f6d977d428bb06b60ed351e338a; no voters:
I20260808 10:44:47.126449 5924 leader_election.cc:288] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:47.126544 5933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:47.126777 5933 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [term 1 LEADER]: Becoming Leader. State: Replica: 4fde9f6d977d428bb06b60ed351e338a, State: Running, Role: LEADER
I20260808 10:44:47.127135 5933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fde9f6d977d428bb06b60ed351e338a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 35245 } }
I20260808 10:44:47.127792 5924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:47.130769 5934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fde9f6d977d428bb06b60ed351e338a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fde9f6d977d428bb06b60ed351e338a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 35245 } } }
I20260808 10:44:47.130744 5935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fde9f6d977d428bb06b60ed351e338a. Latest consensus state: current_term: 1 leader_uuid: "4fde9f6d977d428bb06b60ed351e338a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fde9f6d977d428bb06b60ed351e338a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 35245 } } }
I20260808 10:44:47.130931 5931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:47.115817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.31.237.190:35245)
0808 10:44:47.116247 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:47.116263 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:47.116327 (+ 64us) server_negotiation.cc:407] Connection header received
0808 10:44:47.116978 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:47.116996 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:47.117432 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:47.117816 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:47.118782 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.119975 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.121156 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.121636 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.121796 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:47.124782 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:47.124787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:47.124800 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:47.124826 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:47.126940 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:47.127524 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:47.127528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:47.127533 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:47.127640 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:47.127985 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:47.127993 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:47.127995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:47.128511 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:47.128707 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:47.129498 (+ 791us) server_negotiation.cc:299] Negotiation successful
0808 10:44:47.129825 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":143,"threads_started":1}
I20260808 10:44:47.130993 5934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:47.130950 5935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a [sys.catalog]: This master's current role is: LEADER
W20260808 10:44:47.133854 5951 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:47.133937 5951 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:47.134183 5952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:47.135113 5952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:47.140635 5952 catalog_manager.cc:1347] Generated new cluster ID: 633076f4e2184f78bcba7eea8df49ce9
I20260808 10:44:47.140713 5952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:47.171738 5952 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:47.172818 5952 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:47.177318 5952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fde9f6d977d428bb06b60ed351e338a: Generated new TSK 0
I20260808 10:44:47.178069 5952 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:44:47.212224 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:35245
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:47.338231 5959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:47.338518 5959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:47.338593 5959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:47.342747 5959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:47.342803 5959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:47.342890 5959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:44:47.348192 5959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:35245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:47.349344 5959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:47.350438 5959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:47.360144 5966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:47.360141 5967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:47.360169 5970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:47.360563 5959 server_base.cc:1034] running on GCE node
I20260808 10:44:47.360940 5959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:47.361529 5959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:47.362680 5959 hybrid_clock.cc:648] HybridClock initialized: now 1786185887362666 us; error 28 us; skew 500 ppm
Aug 08 10:44:47 dist-test-slave-twwt krb5kdc[5795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185887, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:47.365574 5959 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:44:47.366504 5959 webserver.cc:460] Webserver started at http://127.31.237.129:46693/ using document root <none> and password file <none>
I20260808 10:44:47.367105 5959 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:47.367169 5959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:47.367427 5959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:47.369540 5959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "3d5b54bfad4f44f4af9bfa26e3cd156b"
format_stamp: "Formatted at 2026-08-08 10:44:47 on dist-test-slave-twwt"
server_key: "2ed715fe4036bb7bb8bcbec8a48e868d"
server_key_iv: "f1acb7e13e896e719d706186bf7c2684"
server_key_version: "encryptionkey@0"
I20260808 10:44:47.370163 5959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "3d5b54bfad4f44f4af9bfa26e3cd156b"
format_stamp: "Formatted at 2026-08-08 10:44:47 on dist-test-slave-twwt"
server_key: "2ed715fe4036bb7bb8bcbec8a48e868d"
server_key_iv: "f1acb7e13e896e719d706186bf7c2684"
server_key_version: "encryptionkey@0"
I20260808 10:44:47.374060 5959 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:44:47.376629 5981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:47.378027 5959 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:47.378149 5959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "3d5b54bfad4f44f4af9bfa26e3cd156b"
format_stamp: "Formatted at 2026-08-08 10:44:47 on dist-test-slave-twwt"
server_key: "2ed715fe4036bb7bb8bcbec8a48e868d"
server_key_iv: "f1acb7e13e896e719d706186bf7c2684"
server_key_version: "encryptionkey@0"
I20260808 10:44:47.378273 5959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:47.398335 5959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:47.402248 5959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:47.402491 5959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:47.403249 5959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:47.404214 5959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:47.404275 5959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:47.404345 5959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:47.404377 5959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:47.413677 5959 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:35625
I20260808 10:44:47.413698 6145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:35625 every 8 connection(s)
I20260808 10:44:47.414644 5959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:44:47.421018 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 5959
I20260808 10:44:47.421164 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:44:47.421473 32694 external_mini_cluster.cc:1428] Setting key 04fd3fd46a1c9151929694e28ea4aca7
Aug 08 10:44:47 dist-test-slave-twwt krb5kdc[5795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185887, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:47.430114 5931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:47.416588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:35243 (local address 127.31.237.190:35245)
0808 10:44:47.416736 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:47.416740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:47.417640 (+ 900us) server_negotiation.cc:407] Connection header received
0808 10:44:47.419132 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:47.419136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:47.419193 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:47.419275 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:47.420577 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.421154 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.422487 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.422815 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.422932 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:47.425668 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:47.425675 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:47.425681 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:47.425716 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:47.427520 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:47.428097 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:47.428104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:47.428109 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:47.428232 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:47.428741 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:47.428748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:47.428751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:47.429018 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:47.429132 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:47.429716 (+ 584us) server_negotiation.cc:299] Negotiation successful
0808 10:44:47.429864 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:44:47.430914 6152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:47.416969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:35245 (local address 127.31.237.129:35243)
0808 10:44:47.417470 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:47.417514 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:47.418910 (+ 1396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:47.419448 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:47.419455 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:47.419845 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:47.420419 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:47.420435 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.421278 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.421281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:47.422356 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:47.422367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.422972 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.422976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:47.423153 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:47.423784 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:47.423800 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:47.425486 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:47.427648 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:47.427652 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:47.427664 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:47.427987 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:47.428354 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:47.428357 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:47.428358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:47.428621 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:47.429130 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:47.429135 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:47.429415 (+ 280us) client_negotiation.cc:769] Sending connection context
0808 10:44:47.429613 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 10:44:47.429941 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":122,"threads_started":1}
I20260808 10:44:47.432168 6149 heartbeater.cc:347] Connected to a master server at 127.31.237.190:35245
I20260808 10:44:47.432406 6149 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:47.432917 6149 heartbeater.cc:511] Master 127.31.237.190:35245 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:44:47.434675 5872 ts_manager.cc:194] Registered new tserver with Master: 3d5b54bfad4f44f4af9bfa26e3cd156b (127.31.237.129:35625)
I20260808 10:44:47.436030 5872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:35243
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:44:47.453617 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:35245
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:47.572467 6155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:47.572863 6155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:47.572960 6155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:47.577186 6155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:47.577289 6155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:47.577394 6155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:44:47.583364 6155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:35245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:47.584573 6155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:47.585737 6155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:47.596309 6164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:47.596376 6155 server_base.cc:1034] running on GCE node
W20260808 10:44:47.596284 6161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:47.596277 6167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:47.597143 6155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:47.597918 6155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:47.599161 6155 hybrid_clock.cc:648] HybridClock initialized: now 1786185887599130 us; error 40 us; skew 500 ppm
Aug 08 10:44:47 dist-test-slave-twwt krb5kdc[5795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185887, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:47.602649 6155 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:44:47.603698 6155 webserver.cc:460] Webserver started at http://127.31.237.130:40401/ using document root <none> and password file <none>
I20260808 10:44:47.604351 6155 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:47.604421 6155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:47.604616 6155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:47.607054 6155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "88a5e6a8880c437eab0f9f114495c822"
format_stamp: "Formatted at 2026-08-08 10:44:47 on dist-test-slave-twwt"
server_key: "b0e913617e1f2d0e2aa918757c4d73e2"
server_key_iv: "ef7c73b39dc8df58c23f8a50d471ac09"
server_key_version: "encryptionkey@0"
I20260808 10:44:47.607652 6155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "88a5e6a8880c437eab0f9f114495c822"
format_stamp: "Formatted at 2026-08-08 10:44:47 on dist-test-slave-twwt"
server_key: "b0e913617e1f2d0e2aa918757c4d73e2"
server_key_iv: "ef7c73b39dc8df58c23f8a50d471ac09"
server_key_version: "encryptionkey@0"
I20260808 10:44:47.612001 6155 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:44:47.614804 6185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:47.616246 6155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:47.616423 6155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "88a5e6a8880c437eab0f9f114495c822"
format_stamp: "Formatted at 2026-08-08 10:44:47 on dist-test-slave-twwt"
server_key: "b0e913617e1f2d0e2aa918757c4d73e2"
server_key_iv: "ef7c73b39dc8df58c23f8a50d471ac09"
server_key_version: "encryptionkey@0"
I20260808 10:44:47.616545 6155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:47.638015 6155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:47.641901 6155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:47.642141 6155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:47.642923 6155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:47.643884 6155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:47.643934 6155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:47.644004 6155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:47.644038 6155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:47.653390 6155 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:35511
I20260808 10:44:47.653410 6337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:35511 every 8 connection(s)
I20260808 10:44:47.654417 6155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:44:47.661902 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 6155
I20260808 10:44:47.662034 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:44:47.662361 32694 external_mini_cluster.cc:1428] Setting key 9ac3394b543507240083325f566759c8
Aug 08 10:44:47 dist-test-slave-twwt krb5kdc[5795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185887, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:47.669480 5931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:47.656484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:46615 (local address 127.31.237.190:35245)
0808 10:44:47.656644 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:47.656648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:47.657543 (+ 895us) server_negotiation.cc:407] Connection header received
0808 10:44:47.659035 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:47.659038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:47.659086 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:47.659164 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:47.660361 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.660892 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.662392 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.662746 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.662868 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:47.665621 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:47.665628 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:47.665634 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:47.665669 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:47.667219 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:47.667843 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:47.667847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:47.667849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:47.667945 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:47.668364 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:47.668369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:47.668371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:47.668565 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:47.668678 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:47.669193 (+ 515us) server_negotiation.cc:299] Negotiation successful
0808 10:44:47.669316 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:44:47.670328 6344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:47.656860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:35245 (local address 127.31.237.130:46615)
0808 10:44:47.657374 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:47.657421 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:47.658824 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:47.659346 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:47.659353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:47.659716 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:47.660209 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:47.660233 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.661035 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.661039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:47.662238 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:47.662250 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.662936 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.662942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:47.663116 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:47.663866 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:47.663888 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:47.665411 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:47.667339 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:47.667348 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:47.667365 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:47.667731 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:47.668044 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:47.668048 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:47.668050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:47.668258 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:47.668680 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:47.668689 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:47.668950 (+ 261us) client_negotiation.cc:769] Sending connection context
0808 10:44:47.669148 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 10:44:47.669496 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":119,"threads_started":1}
I20260808 10:44:47.671581 6338 heartbeater.cc:347] Connected to a master server at 127.31.237.190:35245
I20260808 10:44:47.671854 6338 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:47.672394 6338 heartbeater.cc:511] Master 127.31.237.190:35245 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:44:47.673540 5872 ts_manager.cc:194] Registered new tserver with Master: 88a5e6a8880c437eab0f9f114495c822 (127.31.237.130:35511)
I20260808 10:44:47.674089 5872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:46615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:44:47.692062 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:35245
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:40593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:47.812631 6348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:47.812929 6348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:47.812973 6348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:47.817097 6348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:47.817166 6348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:47.817225 6348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:44:47.823315 6348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:40593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:35245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:47.824498 6348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:47.825575 6348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:47.836158 6359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:47.836170 6354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:47.836141 6364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:47.836220 6348 server_base.cc:1034] running on GCE node
I20260808 10:44:47.836788 6348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:47.837507 6348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:47.838682 6348 hybrid_clock.cc:648] HybridClock initialized: now 1786185887838660 us; error 26 us; skew 500 ppm
Aug 08 10:44:47 dist-test-slave-twwt krb5kdc[5795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185887, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:47.841490 6348 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:44:47.842382 6348 webserver.cc:460] Webserver started at http://127.31.237.131:41189/ using document root <none> and password file <none>
I20260808 10:44:47.843034 6348 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:47.843129 6348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:47.843375 6348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:47.845468 6348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "80e10e0f79a14060b6ecf1c24a682c21"
format_stamp: "Formatted at 2026-08-08 10:44:47 on dist-test-slave-twwt"
server_key: "1081c6a8e3ba287f2a8cf44597edb006"
server_key_iv: "077e1d9a8f76e6209d5d55ab0373ee25"
server_key_version: "encryptionkey@0"
I20260808 10:44:47.846071 6348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "80e10e0f79a14060b6ecf1c24a682c21"
format_stamp: "Formatted at 2026-08-08 10:44:47 on dist-test-slave-twwt"
server_key: "1081c6a8e3ba287f2a8cf44597edb006"
server_key_iv: "077e1d9a8f76e6209d5d55ab0373ee25"
server_key_version: "encryptionkey@0"
I20260808 10:44:47.850001 6348 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:44:47.852772 6375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:47.854086 6348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:44:47.854230 6348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "80e10e0f79a14060b6ecf1c24a682c21"
format_stamp: "Formatted at 2026-08-08 10:44:47 on dist-test-slave-twwt"
server_key: "1081c6a8e3ba287f2a8cf44597edb006"
server_key_iv: "077e1d9a8f76e6209d5d55ab0373ee25"
server_key_version: "encryptionkey@0"
I20260808 10:44:47.854351 6348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:47.873605 6348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:47.876566 6348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:47.876771 6348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:47.877374 6348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:47.878285 6348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:47.878356 6348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:47.878427 6348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:47.878489 6348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:47.887924 6348 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:36609
I20260808 10:44:47.887959 6553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:36609 every 8 connection(s)
I20260808 10:44:47.888878 6348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:44:47.889909 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 6348
I20260808 10:44:47.890056 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:44:47.890427 32694 external_mini_cluster.cc:1428] Setting key 3aabec82c990025500a6de6fbdc79a2c
Aug 08 10:44:47 dist-test-slave-twwt krb5kdc[5795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185887, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:47.903093 5931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:47.891075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:55939 (local address 127.31.237.190:35245)
0808 10:44:47.891211 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:47.891215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:47.892327 (+ 1112us) server_negotiation.cc:407] Connection header received
0808 10:44:47.893885 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:47.893888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:47.893944 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:47.894012 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:47.895271 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.895794 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.896941 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.897161 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.897257 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:47.899819 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:47.899823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:47.899825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:47.899850 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:47.901140 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:47.901616 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:47.901618 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:47.901619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:47.901694 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:47.901995 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:47.901997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:47.901998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:47.902176 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:47.902254 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:47.902776 (+ 522us) server_negotiation.cc:299] Negotiation successful
0808 10:44:47.902877 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:44:47.904203 6557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:47.891597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:35245 (local address 127.31.237.131:55939)
0808 10:44:47.892149 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:47.892209 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:47.893680 (+ 1471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:47.894170 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:47.894177 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:47.894545 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:47.895139 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:47.895154 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.895916 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.895919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:47.896789 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:47.896798 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.897302 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.897305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:47.897447 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:47.898098 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:47.898114 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:47.899660 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:47.901248 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:47.901253 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:47.901265 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:47.901528 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:47.901773 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:47.901775 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:47.901776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:47.901914 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:47.902265 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:47.902271 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:47.902514 (+ 243us) client_negotiation.cc:769] Sending connection context
0808 10:44:47.902737 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 10:44:47.903019 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":142,"threads_started":1}
I20260808 10:44:47.905522 6554 heartbeater.cc:347] Connected to a master server at 127.31.237.190:35245
I20260808 10:44:47.905773 6554 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:47.906286 6554 heartbeater.cc:511] Master 127.31.237.190:35245 requested a full tablet report, sending...
I20260808 10:44:47.907373 5872 ts_manager.cc:194] Registered new tserver with Master: 80e10e0f79a14060b6ecf1c24a682c21 (127.31.237.131:36609)
I20260808 10:44:47.907976 5872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:55939
I20260808 10:44:47.916461 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:44:47 dist-test-slave-twwt krb5kdc[5795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:44:47 dist-test-slave-twwt krb5kdc[5795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.129@KRBTEST.COM
I20260808 10:44:47.944363 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:47.934771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34184 (local address 127.31.237.129:35625)
0808 10:44:47.935093 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:47.935101 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:47.935131 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:44:47.935254 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:47.935269 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:47.935480 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:47.935640 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:47.936335 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.937108 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.937983 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.938352 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.938485 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:47.940864 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:47.940873 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:47.940886 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:47.940912 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:47.942449 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:47.942941 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:47.942948 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:47.942949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:47.943034 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:47.943331 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:47.943335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:47.943336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:47.943594 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:47.943825 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:47.944046 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 10:44:47.944204 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
Aug 08 10:44:47 dist-test-slave-twwt krb5kdc[5795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:47.957819 5931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:47.947262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.31.237.190:35245)
0808 10:44:47.947423 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:47.947427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:47.947443 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:44:47.947565 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:47.947567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:47.947609 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:47.947678 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:47.948241 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.948739 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.949445 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:47.949645 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:47.949720 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:47.953066 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:47.953070 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:47.953073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:47.953103 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:47.955757 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:47.956268 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:47.956271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:47.956273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:47.956352 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:47.956657 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:47.956660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:47.956662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:47.956993 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:47.957151 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:47.957501 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 10:44:47.957643 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 10:44:47.958344 5872 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48776
I20260808 10:44:47.959415 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 5959
I20260808 10:44:47.974083 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 6155
I20260808 10:44:47.985268 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 6348
I20260808 10:44:47.998805 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 5816
2026-08-08T10:44:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2462 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:48 dist-test-slave-twwt krb5kdc[6573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:48 dist-test-slave-twwt krb5kdc[6573](info): set up 2 sockets
Aug 08 10:44:48 dist-test-slave-twwt krb5kdc[6573](info): commencing operation
krb5kdc: starting...
W20260808 10:44:49.245100 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:44:49 dist-test-slave-twwt krb5kdc[6573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:44:49.374830 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:40621
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:40621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:49.489861 6591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:49.490144 6591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:49.490187 6591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:49.494256 6591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:49.494313 6591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:49.494333 6591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:49.494349 6591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:49.494364 6591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:49.499684 6591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:40621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:40621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:49.500780 6591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:49.501854 6591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:49.510125 6599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:49.510147 6596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:49.510133 6601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:49.510195 6591 server_base.cc:1034] running on GCE node
I20260808 10:44:49.511061 6591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:49.512077 6591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:49.513239 6591 hybrid_clock.cc:648] HybridClock initialized: now 1786185889513221 us; error 35 us; skew 500 ppm
Aug 08 10:44:49 dist-test-slave-twwt krb5kdc[6573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185889, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:49.516043 6591 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:44:49.516798 6591 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:49.516870 6591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:49.517134 6591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:49.519191 6591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "07af4779fd69497c99820b11b7f64899"
format_stamp: "Formatted at 2026-08-08 10:44:49 on dist-test-slave-twwt"
server_key: "af5c6d0ab57775c92430d72544c1bd21"
server_key_iv: "a6e95492ef27df5ea148e77c74a2ab23"
server_key_version: "encryptionkey@0"
I20260808 10:44:49.519757 6591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "07af4779fd69497c99820b11b7f64899"
format_stamp: "Formatted at 2026-08-08 10:44:49 on dist-test-slave-twwt"
server_key: "af5c6d0ab57775c92430d72544c1bd21"
server_key_iv: "a6e95492ef27df5ea148e77c74a2ab23"
server_key_version: "encryptionkey@0"
I20260808 10:44:49.523363 6591 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:44:49.525924 6609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:49.527207 6591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:44:49.527336 6591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "07af4779fd69497c99820b11b7f64899"
format_stamp: "Formatted at 2026-08-08 10:44:49 on dist-test-slave-twwt"
server_key: "af5c6d0ab57775c92430d72544c1bd21"
server_key_iv: "a6e95492ef27df5ea148e77c74a2ab23"
server_key_version: "encryptionkey@0"
I20260808 10:44:49.527439 6591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:49.554916 6591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:49.558063 6591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:49.558275 6591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:49.565469 6591 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:40621
I20260808 10:44:49.565752 6682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:40621 every 8 connection(s)
I20260808 10:44:49.566197 6591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:49.569147 6683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:49.572837 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 6591
I20260808 10:44:49.573031 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:49.573318 32694 external_mini_cluster.cc:1428] Setting key 857647209f5d5fe30e1afd0f6eeb970b
I20260808 10:44:49.575645 6683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899: Bootstrap starting.
I20260808 10:44:49.577973 6683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:49.578852 6683 log.cc:824] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:49.580914 6683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899: No bootstrap required, opened a new log
Aug 08 10:44:49 dist-test-slave-twwt krb5kdc[6573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:49.585078 6683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af4779fd69497c99820b11b7f64899" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40621 } }
I20260808 10:44:49.585322 6683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:49.585357 6683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07af4779fd69497c99820b11b7f64899, State: Initialized, Role: FOLLOWER
I20260808 10:44:49.585956 6683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af4779fd69497c99820b11b7f64899" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40621 } }
I20260808 10:44:49.586143 6683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:49.586225 6683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:49.586359 6683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:49.587611 6683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af4779fd69497c99820b11b7f64899" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40621 } }
I20260808 10:44:49.588011 6683 leader_election.cc:302] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07af4779fd69497c99820b11b7f64899; no voters:
I20260808 10:44:49.588363 6683 leader_election.cc:288] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:49.588495 6689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:49.588733 6689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [term 1 LEADER]: Becoming Leader. State: Replica: 07af4779fd69497c99820b11b7f64899, State: Running, Role: LEADER
I20260808 10:44:49.588812 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:49.574856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.31.237.190:40621)
0808 10:44:49.575318 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:49.575334 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:49.575376 (+ 42us) server_negotiation.cc:407] Connection header received
0808 10:44:49.576081 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:49.576102 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:49.576450 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:49.576802 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:49.577653 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:49.578830 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:49.579940 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:49.580436 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:49.580625 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:49.583327 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:49.583333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:49.583351 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:49.583382 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:49.585277 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:49.585868 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:49.585874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:49.585878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:49.585996 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:49.586341 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:49.586344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:49.586346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:49.586936 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:49.587102 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:49.587409 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 10:44:49.587717 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":146,"threads_started":1}
I20260808 10:44:49.589068 6689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af4779fd69497c99820b11b7f64899" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40621 } }
I20260808 10:44:49.589732 6683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:49.590371 6689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07af4779fd69497c99820b11b7f64899. Latest consensus state: current_term: 1 leader_uuid: "07af4779fd69497c99820b11b7f64899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af4779fd69497c99820b11b7f64899" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40621 } } }
I20260808 10:44:49.590492 6689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:49.591427 6699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:49.591442 6691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07af4779fd69497c99820b11b7f64899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af4779fd69497c99820b11b7f64899" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40621 } } }
I20260808 10:44:49.591578 6691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:49.594936 6699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:49.601233 6699 catalog_manager.cc:1347] Generated new cluster ID: 9c355fa191dd46beb74da104dc333c70
I20260808 10:44:49.601325 6699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:49.610954 6699 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:49.611961 6699 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:49.616191 6699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899: Generated new TSK 0
I20260808 10:44:49.616982 6699 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:44:49.654438 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40621
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:49.770928 6721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:49.771204 6721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:49.771250 6721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:49.775343 6721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:49.775414 6721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:49.775477 6721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:44:49.780956 6721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:49.782095 6721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:49.783221 6721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:49.793141 6729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:49.793148 6728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:49.793148 6731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:49.793605 6721 server_base.cc:1034] running on GCE node
I20260808 10:44:49.794070 6721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:49.794910 6721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:49.796079 6721 hybrid_clock.cc:648] HybridClock initialized: now 1786185889796062 us; error 28 us; skew 500 ppm
Aug 08 10:44:49 dist-test-slave-twwt krb5kdc[6573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185889, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:49.802480 6721 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:44:49.803296 6721 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:49.803373 6721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:49.803546 6721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:49.805603 6721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "5a699467562b43dcb2b1d5c422ab14af"
format_stamp: "Formatted at 2026-08-08 10:44:49 on dist-test-slave-twwt"
server_key: "810ff12278b2671e79dc38492b8c96e0"
server_key_iv: "ded610e9518f62a198226177e8ce6120"
server_key_version: "encryptionkey@0"
I20260808 10:44:49.806128 6721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "5a699467562b43dcb2b1d5c422ab14af"
format_stamp: "Formatted at 2026-08-08 10:44:49 on dist-test-slave-twwt"
server_key: "810ff12278b2671e79dc38492b8c96e0"
server_key_iv: "ded610e9518f62a198226177e8ce6120"
server_key_version: "encryptionkey@0"
I20260808 10:44:49.809955 6721 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260808 10:44:49.812489 6742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:49.813687 6721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:44:49.813812 6721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "5a699467562b43dcb2b1d5c422ab14af"
format_stamp: "Formatted at 2026-08-08 10:44:49 on dist-test-slave-twwt"
server_key: "810ff12278b2671e79dc38492b8c96e0"
server_key_iv: "ded610e9518f62a198226177e8ce6120"
server_key_version: "encryptionkey@0"
I20260808 10:44:49.813917 6721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:49.833809 6721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:49.836882 6721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:49.837124 6721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:49.837764 6721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:49.838678 6721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:49.838757 6721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:49.838836 6721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:49.838891 6721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:49.848062 6721 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:33829
I20260808 10:44:49.848085 6919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:33829 every 8 connection(s)
I20260808 10:44:49.848691 6721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:44:49.851763 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 6721
I20260808 10:44:49.851912 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:44:49.852262 32694 external_mini_cluster.cc:1428] Setting key ab25db0852984d3453f6126301a6bcca
Aug 08 10:44:49 dist-test-slave-twwt krb5kdc[6573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185889, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:44:49.864578 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:49.850513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:55457 (local address 127.31.237.190:40621)
0808 10:44:49.850718 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:49.850723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:49.851706 (+ 983us) server_negotiation.cc:407] Connection header received
0808 10:44:49.853391 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:49.853395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:49.853439 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:49.853530 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:49.855265 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:49.855788 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:49.857057 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:49.857469 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:49.857594 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:49.860444 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:49.860451 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:49.860457 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:49.860494 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:49.862276 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:49.862963 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:49.862968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:49.862970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:49.863069 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:49.863444 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:49.863448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:49.863450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:49.863684 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:49.863788 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:49.864352 (+ 564us) server_negotiation.cc:299] Negotiation successful
0808 10:44:49.864456 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:44:49.865466 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:49.850972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:40621 (local address 127.31.237.129:55457)
0808 10:44:49.851505 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:49.851569 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:49.853168 (+ 1599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:49.853748 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:49.853765 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:49.854329 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:49.855104 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:49.855127 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:49.855951 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:49.855957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:49.856926 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:49.856937 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:49.857618 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:49.857621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:49.857771 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:49.858442 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:49.858458 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:49.860251 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:49.862408 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:49.862413 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:49.862426 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:49.862834 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:49.863172 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:49.863175 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:49.863176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:49.863322 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:49.863810 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:49.863815 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:49.864101 (+ 286us) client_negotiation.cc:769] Sending connection context
0808 10:44:49.864266 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 10:44:49.864523 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":117,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
I20260808 10:44:49.867054 6920 heartbeater.cc:347] Connected to a master server at 127.31.237.190:40621
I20260808 10:44:49.867319 6920 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:49.867795 6920 heartbeater.cc:511] Master 127.31.237.190:40621 requested a full tablet report, sending...
I20260808 10:44:49.869284 6632 ts_manager.cc:194] Registered new tserver with Master: 5a699467562b43dcb2b1d5c422ab14af (127.31.237.129:33829)
I20260808 10:44:49.870429 6632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:55457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:44:49.882606 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40621
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:49.999758 6926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:50.000012 6926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:50.000068 6926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:50.004062 6926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:50.004123 6926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:50.004227 6926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:44:50.010139 6926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:50.011636 6926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:50.013095 6926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:50.023149 6938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:50.023137 6935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:50.023145 6934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:50.023198 6926 server_base.cc:1034] running on GCE node
I20260808 10:44:50.023810 6926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:50.024526 6926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:50.025712 6926 hybrid_clock.cc:648] HybridClock initialized: now 1786185890025676 us; error 48 us; skew 500 ppm
Aug 08 10:44:50 dist-test-slave-twwt krb5kdc[6573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185890, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:50.028765 6926 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:44:50.029580 6926 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:50.029646 6926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:50.029881 6926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:50.032306 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "6ae5d3e0513f459386c3d17008078bdf"
format_stamp: "Formatted at 2026-08-08 10:44:50 on dist-test-slave-twwt"
server_key: "84bc5a74ce46b296aa8451a2a9536646"
server_key_iv: "0321e05a9ea044a7b3c1b3ac766a0608"
server_key_version: "encryptionkey@0"
I20260808 10:44:50.032924 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "6ae5d3e0513f459386c3d17008078bdf"
format_stamp: "Formatted at 2026-08-08 10:44:50 on dist-test-slave-twwt"
server_key: "84bc5a74ce46b296aa8451a2a9536646"
server_key_iv: "0321e05a9ea044a7b3c1b3ac766a0608"
server_key_version: "encryptionkey@0"
I20260808 10:44:50.036792 6926 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:44:50.039467 6950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:50.040737 6926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:44:50.040869 6926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "6ae5d3e0513f459386c3d17008078bdf"
format_stamp: "Formatted at 2026-08-08 10:44:50 on dist-test-slave-twwt"
server_key: "84bc5a74ce46b296aa8451a2a9536646"
server_key_iv: "0321e05a9ea044a7b3c1b3ac766a0608"
server_key_version: "encryptionkey@0"
I20260808 10:44:50.040978 6926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:50.060793 6926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:50.063925 6926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:50.064095 6926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:50.064663 6926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:50.065493 6926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:50.065539 6926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:50.065582 6926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:50.065598 6926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:50.075237 6926 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:34561
I20260808 10:44:50.075258 7118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:34561 every 8 connection(s)
I20260808 10:44:50.075947 6926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:44:50.079646 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 6926
I20260808 10:44:50.079782 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:44:50.080097 32694 external_mini_cluster.cc:1428] Setting key ae96705ee46c98bc80ae7b8883794c6c
Aug 08 10:44:50 dist-test-slave-twwt krb5kdc[6573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185890, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:44:50.092242 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.077940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:38415 (local address 127.31.237.190:40621)
0808 10:44:50.078089 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:50.078093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:50.079045 (+ 952us) server_negotiation.cc:407] Connection header received
0808 10:44:50.080918 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:50.080922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:50.080983 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:50.081066 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:50.082487 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.083108 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.084811 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.085038 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.085130 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.087898 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:50.087921 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:50.087926 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:50.087965 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:50.089693 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:50.090378 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:50.090384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:50.090386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:50.090475 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:50.090982 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:50.090989 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:50.090993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:50.091200 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:50.091284 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:50.092002 (+ 718us) server_negotiation.cc:299] Negotiation successful
0808 10:44:50.092110 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:44:50.093269 7122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.078342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:40621 (local address 127.31.237.130:38415)
0808 10:44:50.078880 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:50.078930 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:50.080608 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:50.081261 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:50.081268 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:50.081795 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:50.082321 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:50.082335 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.083251 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.083260 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:50.084649 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:50.084667 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.085211 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.085218 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:50.085444 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.086082 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:50.086099 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:50.087695 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:50.089818 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:50.089825 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:50.089850 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:50.090255 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:50.090579 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:50.090583 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:50.090587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:50.090860 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:50.091318 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:50.091325 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:50.091686 (+ 361us) client_negotiation.cc:769] Sending connection context
0808 10:44:50.091924 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 10:44:50.092218 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":111,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
I20260808 10:44:50.094700 7119 heartbeater.cc:347] Connected to a master server at 127.31.237.190:40621
I20260808 10:44:50.095005 7119 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:50.095664 7119 heartbeater.cc:511] Master 127.31.237.190:40621 requested a full tablet report, sending...
I20260808 10:44:50.096962 6632 ts_manager.cc:194] Registered new tserver with Master: 6ae5d3e0513f459386c3d17008078bdf (127.31.237.130:34561)
I20260808 10:44:50.097817 6632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:38415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:44:50.111938 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40621
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:50.227538 7125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:50.227807 7125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:50.227854 7125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:50.232208 7125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:50.232272 7125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:50.232383 7125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:44:50.237900 7125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:50.239099 7125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:50.240170 7125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:50.250190 7134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:50.250335 7135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:50.250411 7125 server_base.cc:1034] running on GCE node
W20260808 10:44:50.250561 7138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:50.251091 7125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:50.251724 7125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:50.252890 7125 hybrid_clock.cc:648] HybridClock initialized: now 1786185890252875 us; error 26 us; skew 500 ppm
Aug 08 10:44:50 dist-test-slave-twwt krb5kdc[6573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185890, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:50.255784 7125 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:44:50.256549 7125 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:50.256649 7125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:50.256917 7125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:50.259184 7125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "c6e1746f48114bdd825c2c0e992d11af"
format_stamp: "Formatted at 2026-08-08 10:44:50 on dist-test-slave-twwt"
server_key: "e93e9591f2db75717706d3a3ac43a0a2"
server_key_iv: "dfd1f6530377d10d44b55b81511b32cd"
server_key_version: "encryptionkey@0"
I20260808 10:44:50.259766 7125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "c6e1746f48114bdd825c2c0e992d11af"
format_stamp: "Formatted at 2026-08-08 10:44:50 on dist-test-slave-twwt"
server_key: "e93e9591f2db75717706d3a3ac43a0a2"
server_key_iv: "dfd1f6530377d10d44b55b81511b32cd"
server_key_version: "encryptionkey@0"
I20260808 10:44:50.263707 7125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:44:50.266211 7147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:50.267472 7125 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:44:50.267617 7125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "c6e1746f48114bdd825c2c0e992d11af"
format_stamp: "Formatted at 2026-08-08 10:44:50 on dist-test-slave-twwt"
server_key: "e93e9591f2db75717706d3a3ac43a0a2"
server_key_iv: "dfd1f6530377d10d44b55b81511b32cd"
server_key_version: "encryptionkey@0"
I20260808 10:44:50.267736 7125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:50.299535 7125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:50.302728 7125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:50.302961 7125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:50.303601 7125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:50.304502 7125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:50.304574 7125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:50.304651 7125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:50.304702 7125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:50.313992 7125 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:39719
I20260808 10:44:50.314010 7311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:39719 every 8 connection(s)
I20260808 10:44:50.314599 7125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:44:50.319458 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 7125
I20260808 10:44:50.319597 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:44:50.319893 32694 external_mini_cluster.cc:1428] Setting key c314bfbbd8f15f5b5d2cf98986698a88
Aug 08 10:44:50 dist-test-slave-twwt krb5kdc[6573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185890, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:50.328794 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.316427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:42885 (local address 127.31.237.190:40621)
0808 10:44:50.316581 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:50.316588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:50.317438 (+ 850us) server_negotiation.cc:407] Connection header received
0808 10:44:50.318988 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:50.318993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:50.319056 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:50.319155 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:50.320702 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.321465 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.322868 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.323087 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.323168 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.325477 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:50.325480 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:50.325482 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:50.325506 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:50.326876 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:50.327343 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:50.327345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:50.327346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:50.327419 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:50.327775 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:50.327777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:50.327779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:50.327955 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:50.328029 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:50.328581 (+ 552us) server_negotiation.cc:299] Negotiation successful
0808 10:44:50.328675 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:44:50.329645 7315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.316832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:40621 (local address 127.31.237.131:42885)
0808 10:44:50.317282 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:50.317326 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:50.318765 (+ 1439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:50.319368 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:50.319376 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:50.319855 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:50.320508 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:50.320548 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.321589 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.321592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:50.322763 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:50.322773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.323216 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.323219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:50.323366 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.323869 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:50.323885 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:50.325310 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:50.326990 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:50.326995 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:50.327006 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:50.327257 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:50.327525 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:50.327534 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:50.327536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:50.327688 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:50.328045 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:50.328050 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:50.328309 (+ 259us) client_negotiation.cc:769] Sending connection context
0808 10:44:50.328529 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 10:44:50.328781 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":112,"threads_started":1}
I20260808 10:44:50.330897 7312 heartbeater.cc:347] Connected to a master server at 127.31.237.190:40621
I20260808 10:44:50.331151 7312 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:50.331640 7312 heartbeater.cc:511] Master 127.31.237.190:40621 requested a full tablet report, sending...
I20260808 10:44:50.332698 6632 ts_manager.cc:194] Registered new tserver with Master: c6e1746f48114bdd825c2c0e992d11af (127.31.237.131:39719)
I20260808 10:44:50.333441 6632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:42885
I20260808 10:44:50.333935 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:44:50.343387 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.335992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36826 (local address 127.31.237.190:40621)
0808 10:44:50.336109 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:50.336115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:50.336312 (+ 197us) server_negotiation.cc:407] Connection header received
0808 10:44:50.336417 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:50.336419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:50.336456 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:50.336520 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:50.337225 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.337704 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.338567 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.338885 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.338992 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.340323 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:50.340327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:50.340329 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:50.340351 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:50.341778 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:50.342170 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:50.342173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:50.342174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:50.342253 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:50.342517 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:50.342520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:50.342521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:50.342810 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:50.342938 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:50.343171 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 10:44:50.343270 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:44:50.346729 6632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:44:50.349810 6632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:44:50.365033 7338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.359209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50486 (local address 127.31.237.131:39719)
0808 10:44:50.359595 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:50.359604 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:50.359825 (+ 221us) server_negotiation.cc:407] Connection header received
0808 10:44:50.359937 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:50.359941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:50.360127 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:50.360285 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:50.360865 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.361614 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.363422 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.364237 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.364374 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.364519 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:50.364744 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 10:44:50.364853 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":116,"threads_started":1}
I20260808 10:44:50.365029 7337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.358775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:39719 (local address 127.0.0.1:50486)
0808 10:44:50.359702 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:50.359712 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:50.359832 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:50.360483 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:50.360490 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:50.360526 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:50.360715 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:50.360723 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.361772 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.361778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:50.363282 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:50.363293 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.364360 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.364364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:50.364522 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.364581 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 10:44:50.364703 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 10:44:50.364864 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"mutex_wait_us":177,"spinlock_wait_cycles":5632,"thread_start_us":94,"threads_started":1}
I20260808 10:44:50.365068 7333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.358366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:34561 (local address 127.0.0.1:55628)
0808 10:44:50.359038 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:50.359089 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:50.359235 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:50.360521 (+ 1286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:50.360524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:50.360543 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:50.360732 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:50.360740 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.362257 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.362260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:50.363421 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:50.363437 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.364400 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.364404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:50.364567 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.364592 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:44:50.364675 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:44:50.364886 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":109,"thread_start_us":71,"threads_started":1}
I20260808 10:44:50.365589 7335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.358507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55628 (local address 127.31.237.130:34561)
0808 10:44:50.360101 (+ 1594us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:50.360114 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:50.360138 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:44:50.360219 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:50.360235 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:50.360381 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:50.360542 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:50.360888 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.362059 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.363544 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.364250 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.365161 (+ 911us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.365282 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:50.365330 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 10:44:50.365435 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1483,"thread_start_us":130,"threads_started":1}
I20260808 10:44:50.365602 7334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.358408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:33829 (local address 127.0.0.1:37984)
0808 10:44:50.359221 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:50.359236 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:50.359358 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:50.360483 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:50.360490 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:50.360527 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:50.360715 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:50.360723 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.362193 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.362198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:50.363931 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:50.363943 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.365206 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.365210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:50.365300 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.365322 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:44:50.365412 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 10:44:50.365471 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":23,"spinlock_wait_cycles":12032,"thread_start_us":97,"threads_started":1}
I20260808 10:44:50.365914 7339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.359219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37984 (local address 127.31.237.129:33829)
0808 10:44:50.359787 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:50.359796 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:50.359818 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:44:50.359912 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:50.359918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:50.360127 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:50.360285 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:50.360853 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.362035 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.364082 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.364897 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.365017 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.365141 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:50.365491 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 10:44:50.365595 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":105,"threads_started":1}
I20260808 10:44:50.369395 6817 tablet_service.cc:1467] Processing CreateTablet for tablet 6c95451e224e48b4a306a2ae8e55a27d (DEFAULT_TABLE table=test-table [id=5d0d42ede8644088ac43b0eb8890ec64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:50.369395 7022 tablet_service.cc:1467] Processing CreateTablet for tablet 6c95451e224e48b4a306a2ae8e55a27d (DEFAULT_TABLE table=test-table [id=5d0d42ede8644088ac43b0eb8890ec64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:50.370668 6817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c95451e224e48b4a306a2ae8e55a27d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:50.370668 7022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c95451e224e48b4a306a2ae8e55a27d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:50.372367 7215 tablet_service.cc:1467] Processing CreateTablet for tablet 6c95451e224e48b4a306a2ae8e55a27d (DEFAULT_TABLE table=test-table [id=5d0d42ede8644088ac43b0eb8890ec64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:50.373435 7215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c95451e224e48b4a306a2ae8e55a27d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:50.378657 7340 tablet_bootstrap.cc:492] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af: Bootstrap starting.
I20260808 10:44:50.380074 7341 tablet_bootstrap.cc:492] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf: Bootstrap starting.
I20260808 10:44:50.380206 7343 tablet_bootstrap.cc:492] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af: Bootstrap starting.
I20260808 10:44:50.381302 7340 tablet_bootstrap.cc:654] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:50.382083 7341 tablet_bootstrap.cc:654] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:50.382302 7343 tablet_bootstrap.cc:654] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:50.382261 7340 log.cc:824] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:50.383453 7341 log.cc:824] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:50.383612 7343 log.cc:824] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:50.384464 7340 tablet_bootstrap.cc:492] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af: No bootstrap required, opened a new log
I20260808 10:44:50.384677 7340 ts_tablet_manager.cc:1397] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:44:50.385269 7341 tablet_bootstrap.cc:492] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf: No bootstrap required, opened a new log
I20260808 10:44:50.385469 7341 ts_tablet_manager.cc:1397] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:44:50.388067 7343 tablet_bootstrap.cc:492] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af: No bootstrap required, opened a new log
I20260808 10:44:50.388254 7343 ts_tablet_manager.cc:1397] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260808 10:44:50.388674 7341 raft_consensus.cc:348] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae5d3e0513f459386c3d17008078bdf" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34561 } } peers { permanent_uuid: "5a699467562b43dcb2b1d5c422ab14af" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 33829 } } peers { permanent_uuid: "c6e1746f48114bdd825c2c0e992d11af" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39719 } }
I20260808 10:44:50.388720 7340 raft_consensus.cc:348] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae5d3e0513f459386c3d17008078bdf" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34561 } } peers { permanent_uuid: "5a699467562b43dcb2b1d5c422ab14af" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 33829 } } peers { permanent_uuid: "c6e1746f48114bdd825c2c0e992d11af" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39719 } }
I20260808 10:44:50.388954 7341 raft_consensus.cc:374] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:50.389000 7340 raft_consensus.cc:374] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:50.389007 7341 raft_consensus.cc:729] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae5d3e0513f459386c3d17008078bdf, State: Initialized, Role: FOLLOWER
I20260808 10:44:50.389063 7340 raft_consensus.cc:729] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a699467562b43dcb2b1d5c422ab14af, State: Initialized, Role: FOLLOWER
I20260808 10:44:50.389477 7341 consensus_queue.cc:260] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae5d3e0513f459386c3d17008078bdf" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34561 } } peers { permanent_uuid: "5a699467562b43dcb2b1d5c422ab14af" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 33829 } } peers { permanent_uuid: "c6e1746f48114bdd825c2c0e992d11af" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39719 } }
I20260808 10:44:50.389599 7340 consensus_queue.cc:260] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae5d3e0513f459386c3d17008078bdf" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34561 } } peers { permanent_uuid: "5a699467562b43dcb2b1d5c422ab14af" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 33829 } } peers { permanent_uuid: "c6e1746f48114bdd825c2c0e992d11af" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39719 } }
I20260808 10:44:50.390375 7119 heartbeater.cc:503] Master 127.31.237.190:40621 was elected leader, sending a full tablet report...
I20260808 10:44:50.390594 7341 ts_tablet_manager.cc:1428] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 10:44:50.390672 7340 ts_tablet_manager.cc:1428] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:44:50.391011 6920 heartbeater.cc:503] Master 127.31.237.190:40621 was elected leader, sending a full tablet report...
I20260808 10:44:50.392426 7343 raft_consensus.cc:348] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae5d3e0513f459386c3d17008078bdf" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34561 } } peers { permanent_uuid: "5a699467562b43dcb2b1d5c422ab14af" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 33829 } } peers { permanent_uuid: "c6e1746f48114bdd825c2c0e992d11af" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39719 } }
I20260808 10:44:50.392740 7343 raft_consensus.cc:374] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:50.392819 7343 raft_consensus.cc:729] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6e1746f48114bdd825c2c0e992d11af, State: Initialized, Role: FOLLOWER
I20260808 10:44:50.393337 7343 consensus_queue.cc:260] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae5d3e0513f459386c3d17008078bdf" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34561 } } peers { permanent_uuid: "5a699467562b43dcb2b1d5c422ab14af" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 33829 } } peers { permanent_uuid: "c6e1746f48114bdd825c2c0e992d11af" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39719 } }
I20260808 10:44:50.394002 7312 heartbeater.cc:503] Master 127.31.237.190:40621 was elected leader, sending a full tablet report...
I20260808 10:44:50.394358 7343 ts_tablet_manager.cc:1428] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:44:50.526284 7350 raft_consensus.cc:482] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:44:50.526526 7350 raft_consensus.cc:504] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae5d3e0513f459386c3d17008078bdf" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34561 } } peers { permanent_uuid: "5a699467562b43dcb2b1d5c422ab14af" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 33829 } } peers { permanent_uuid: "c6e1746f48114bdd825c2c0e992d11af" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39719 } }
I20260808 10:44:50.527832 7350 leader_election.cc:288] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ae5d3e0513f459386c3d17008078bdf (127.31.237.130:34561), c6e1746f48114bdd825c2c0e992d11af (127.31.237.131:39719)
I20260808 10:44:50.532311 7335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.528210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:56629 (local address 127.31.237.130:34561)
0808 10:44:50.528342 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:50.528347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:50.528714 (+ 367us) server_negotiation.cc:407] Connection header received
0808 10:44:50.528901 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:50.528904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:50.528949 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:50.529025 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:50.529305 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.529849 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.531275 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.531680 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.531775 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.531838 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:50.532101 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 10:44:50.532180 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:44:50.532335 7353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.528105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:39719 (local address 127.31.237.129:43133)
0808 10:44:50.528517 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:50.528530 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:50.528709 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:50.529162 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:50.529164 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:50.529179 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:50.529323 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:50.529330 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.530089 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.530092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:50.531022 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:50.531033 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.531912 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.531916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:50.532013 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.532063 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 10:44:50.532144 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 10:44:50.532206 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":157,"threads_started":1}
I20260808 10:44:50.532366 7338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.528210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:43133 (local address 127.31.237.131:39719)
0808 10:44:50.528353 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:50.528359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:50.528618 (+ 259us) server_negotiation.cc:407] Connection header received
0808 10:44:50.528887 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:50.528891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:50.528945 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:50.529000 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:50.529428 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.529967 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.531200 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.531765 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.531889 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.531945 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:50.532162 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 10:44:50.532249 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:44:50.532343 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.528105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:34561 (local address 127.31.237.129:56629)
0808 10:44:50.528621 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:50.528634 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:50.528769 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:50.529048 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:50.529050 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:50.529065 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:50.529175 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:50.529180 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.529961 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.529965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:50.531140 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:50.531155 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.531800 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.531806 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:50.531909 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.531973 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 10:44:50.532071 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 10:44:50.532155 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":98,"thread_start_us":128,"threads_started":1}
I20260808 10:44:50.533536 7247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c95451e224e48b4a306a2ae8e55a27d" candidate_uuid: "5a699467562b43dcb2b1d5c422ab14af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6e1746f48114bdd825c2c0e992d11af" is_pre_election: true
I20260808 10:44:50.533537 7054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c95451e224e48b4a306a2ae8e55a27d" candidate_uuid: "5a699467562b43dcb2b1d5c422ab14af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ae5d3e0513f459386c3d17008078bdf" is_pre_election: true
I20260808 10:44:50.533867 7247 raft_consensus.cc:2454] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a699467562b43dcb2b1d5c422ab14af in term 0.
I20260808 10:44:50.533867 7054 raft_consensus.cc:2454] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a699467562b43dcb2b1d5c422ab14af in term 0.
I20260808 10:44:50.534359 6746 leader_election.cc:302] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a699467562b43dcb2b1d5c422ab14af, c6e1746f48114bdd825c2c0e992d11af; no voters:
I20260808 10:44:50.534677 7350 raft_consensus.cc:2781] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:44:50.534839 7350 raft_consensus.cc:482] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:44:50.534907 7350 raft_consensus.cc:3037] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:50.536005 7350 raft_consensus.cc:504] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae5d3e0513f459386c3d17008078bdf" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34561 } } peers { permanent_uuid: "5a699467562b43dcb2b1d5c422ab14af" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 33829 } } peers { permanent_uuid: "c6e1746f48114bdd825c2c0e992d11af" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39719 } }
I20260808 10:44:50.536371 7350 leader_election.cc:288] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [CANDIDATE]: Term 1 election: Requested vote from peers 6ae5d3e0513f459386c3d17008078bdf (127.31.237.130:34561), c6e1746f48114bdd825c2c0e992d11af (127.31.237.131:39719)
I20260808 10:44:50.536792 7054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c95451e224e48b4a306a2ae8e55a27d" candidate_uuid: "5a699467562b43dcb2b1d5c422ab14af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ae5d3e0513f459386c3d17008078bdf"
I20260808 10:44:50.536806 7247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c95451e224e48b4a306a2ae8e55a27d" candidate_uuid: "5a699467562b43dcb2b1d5c422ab14af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6e1746f48114bdd825c2c0e992d11af"
I20260808 10:44:50.536929 7247 raft_consensus.cc:3037] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:50.536929 7054 raft_consensus.cc:3037] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:50.538245 7054 raft_consensus.cc:2454] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a699467562b43dcb2b1d5c422ab14af in term 1.
I20260808 10:44:50.538245 7247 raft_consensus.cc:2454] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a699467562b43dcb2b1d5c422ab14af in term 1.
I20260808 10:44:50.538589 6744 leader_election.cc:302] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5a699467562b43dcb2b1d5c422ab14af, 6ae5d3e0513f459386c3d17008078bdf, c6e1746f48114bdd825c2c0e992d11af; no voters:
I20260808 10:44:50.538863 7350 raft_consensus.cc:2781] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:50.539141 7350 raft_consensus.cc:686] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [term 1 LEADER]: Becoming Leader. State: Replica: 5a699467562b43dcb2b1d5c422ab14af, State: Running, Role: LEADER
I20260808 10:44:50.539518 7350 consensus_queue.cc:237] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae5d3e0513f459386c3d17008078bdf" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34561 } } peers { permanent_uuid: "5a699467562b43dcb2b1d5c422ab14af" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 33829 } } peers { permanent_uuid: "c6e1746f48114bdd825c2c0e992d11af" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39719 } }
I20260808 10:44:50.542776 6629 catalog_manager.cc:5495] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a699467562b43dcb2b1d5c422ab14af (127.31.237.129). New cstate: current_term: 1 leader_uuid: "5a699467562b43dcb2b1d5c422ab14af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae5d3e0513f459386c3d17008078bdf" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a699467562b43dcb2b1d5c422ab14af" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 33829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6e1746f48114bdd825c2c0e992d11af" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39719 } health_report { overall_health: UNKNOWN } } }
I20260808 10:44:50.555219 7339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.551727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37988 (local address 127.31.237.129:33829)
0808 10:44:50.551894 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:50.551898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:50.551913 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:44:50.551992 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:50.551995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:50.552054 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:50.552147 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:44:50.552394 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.552944 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.553774 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:50.554001 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:50.554089 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:50.554373 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:44:50.554891 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:44:50.554997 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:50.555059 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 10:44:50.555111 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 10:44:50.561527 7247 raft_consensus.cc:1261] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af [term 1 FOLLOWER]: Refusing update from remote peer 5a699467562b43dcb2b1d5c422ab14af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:50.561545 7054 raft_consensus.cc:1261] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf [term 1 FOLLOWER]: Refusing update from remote peer 5a699467562b43dcb2b1d5c422ab14af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:50.562256 7355 consensus_queue.cc:1035] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ae5d3e0513f459386c3d17008078bdf" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:50.562459 7350 consensus_queue.cc:1035] T 6c95451e224e48b4a306a2ae8e55a27d P 5a699467562b43dcb2b1d5c422ab14af [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6e1746f48114bdd825c2c0e992d11af" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:44:50.565646 7314 tablet.cc:2369] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:44:50.569312 7359 mvcc.cc:205] Tried to move back new op lower bound from 7316217407732736000 to 7316217407652573184. Current Snapshot: MvccSnapshot[applied={T|T < 7316217407732736000}]
I20260808 10:44:50.582513 7361 mvcc.cc:205] Tried to move back new op lower bound from 7316217407732736000 to 7316217407652573184. Current Snapshot: MvccSnapshot[applied={T|T < 7316217407732736000}]
I20260808 10:44:50.587790 6630 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:50.587991 6630 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:50.590495 6630 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 07af4779fd69497c99820b11b7f64899: Sending DeleteTablet for 3 replicas of tablet 6c95451e224e48b4a306a2ae8e55a27d
I20260808 10:44:50.591450 6817 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c95451e224e48b4a306a2ae8e55a27d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:50 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:37984
I20260808 10:44:50.591493 7215 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c95451e224e48b4a306a2ae8e55a27d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:50 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:50486
I20260808 10:44:50.591552 7022 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c95451e224e48b4a306a2ae8e55a27d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:50 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:55628
I20260808 10:44:50.592016 7378 tablet_replica.cc:330] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af: stopping tablet replica
I20260808 10:44:50.592165 7377 tablet_replica.cc:330] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf: stopping tablet replica
I20260808 10:44:50.592370 7378 raft_consensus.cc:2229] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:50.592399 7377 raft_consensus.cc:2229] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:50.592711 7378 raft_consensus.cc:2258] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:50.592710 7377 raft_consensus.cc:2258] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:50.592974 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 6721
I20260808 10:44:50.596978 7378 ts_tablet_manager.cc:1905] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:50.599872 7378 ts_tablet_manager.cc:1918] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:44:50.599972 7378 log.cc:1196] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/6c95451e224e48b4a306a2ae8e55a27d
I20260808 10:44:50.600323 7378 ts_tablet_manager.cc:1939] T 6c95451e224e48b4a306a2ae8e55a27d P c6e1746f48114bdd825c2c0e992d11af: Deleting consensus metadata
I20260808 10:44:50.600339 7377 ts_tablet_manager.cc:1905] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:50.602263 6615 catalog_manager.cc:4841] TS c6e1746f48114bdd825c2c0e992d11af (127.31.237.131:39719): tablet 6c95451e224e48b4a306a2ae8e55a27d (table test-table [id=5d0d42ede8644088ac43b0eb8890ec64]) successfully deleted
I20260808 10:44:50.603034 7377 ts_tablet_manager.cc:1918] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:44:50.603156 7377 log.cc:1196] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/6c95451e224e48b4a306a2ae8e55a27d
I20260808 10:44:50.603457 7377 ts_tablet_manager.cc:1939] T 6c95451e224e48b4a306a2ae8e55a27d P 6ae5d3e0513f459386c3d17008078bdf: Deleting consensus metadata
I20260808 10:44:50.604594 6611 catalog_manager.cc:4841] TS 6ae5d3e0513f459386c3d17008078bdf (127.31.237.130:34561): tablet 6c95451e224e48b4a306a2ae8e55a27d (table test-table [id=5d0d42ede8644088ac43b0eb8890ec64]) successfully deleted
W20260808 10:44:50.606595 6610 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:33829 (error 108)
I20260808 10:44:50.607522 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 6926
I20260808 10:44:50.607748 7337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.607391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:33829 (local address 127.0.0.1:37992)
0808 10:44:50.607553 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:50.607640 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:33829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 10:44:50.614918 6610 catalog_manager.cc:4584] TS 5a699467562b43dcb2b1d5c422ab14af (127.31.237.129:33829): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6c95451e224e48b4a306a2ae8e55a27d: Network error: Client connection negotiation failed: client connection to 127.31.237.129:33829: connect: Connection refused (error 111)
I20260808 10:44:50.620793 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 7125
I20260808 10:44:50.634183 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 6591
2026-08-08T10:44:50Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2633 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T10:44:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:50Z Disabled control of system clock
I20260808 10:44:50.673121 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:36923
--webserver_interface=127.31.237.190
--webserver_port=0
--builtin_ntp_servers=127.31.237.148:42983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:36923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:44:50.806473 7384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:50.806967 7384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:50.807034 7384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:50.812919 7384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:50.813011 7384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:50.813033 7384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:50.813057 7384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:50.813086 7384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:50.819468 7384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:36923
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:36923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:50.820696 7384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:50.822203 7384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:50.831302 7392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:50.831301 7391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:50.831328 7395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:50.831362 7384 server_base.cc:1034] running on GCE node
I20260808 10:44:50.832196 7384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:50.833326 7384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:50.834529 7384 hybrid_clock.cc:648] HybridClock initialized: now 1786185890834516 us; error 26 us; skew 500 ppm
I20260808 10:44:50.836545 7384 webserver.cc:460] Webserver started at http://127.31.237.190:34507/ using document root <none> and password file <none>
I20260808 10:44:50.837146 7384 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:50.837242 7384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:50.837499 7384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:50.839800 7384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "8d40648583c04eb2bb98c171854d8439"
format_stamp: "Formatted at 2026-08-08 10:44:50 on dist-test-slave-twwt"
server_key: "daba5f003795f7e124182b704649980a"
server_key_iv: "33e33022fc3c7fab4648a92ae015d6da"
server_key_version: "encryptionkey@0"
I20260808 10:44:50.840390 7384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "8d40648583c04eb2bb98c171854d8439"
format_stamp: "Formatted at 2026-08-08 10:44:50 on dist-test-slave-twwt"
server_key: "daba5f003795f7e124182b704649980a"
server_key_iv: "33e33022fc3c7fab4648a92ae015d6da"
server_key_version: "encryptionkey@0"
I20260808 10:44:50.844125 7384 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:44:50.846639 7403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:50.847908 7384 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:50.848053 7384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "8d40648583c04eb2bb98c171854d8439"
format_stamp: "Formatted at 2026-08-08 10:44:50 on dist-test-slave-twwt"
server_key: "daba5f003795f7e124182b704649980a"
server_key_iv: "33e33022fc3c7fab4648a92ae015d6da"
server_key_version: "encryptionkey@0"
I20260808 10:44:50.848171 7384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:50.852988 7384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:50.853607 7384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:50.853799 7384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:50.860946 7384 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:36923
I20260808 10:44:50.860970 7483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:36923 every 8 connection(s)
I20260808 10:44:50.861938 7384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:50.865293 7484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:50.870078 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 7384
I20260808 10:44:50.870182 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:50.870497 32694 external_mini_cluster.cc:1428] Setting key f090752a1dbfddcb0e32015a6c63b220
I20260808 10:44:50.871724 7484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439: Bootstrap starting.
I20260808 10:44:50.874083 7484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:50.875007 7484 log.cc:824] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:50.876309 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:50.872034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47684 (local address 127.31.237.190:36923)
0808 10:44:50.872703 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:50.872730 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:50.872767 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:44:50.873411 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:50.873429 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:50.873780 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:50.874136 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:50.874986 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:50.874991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:50.875003 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:50.875014 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:50.875071 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:50.875087 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:50.875213 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:50.875381 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 10:44:50.875634 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":146,"threads_started":1}
I20260808 10:44:50.877175 7484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439: No bootstrap required, opened a new log
I20260808 10:44:50.880795 7484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d40648583c04eb2bb98c171854d8439" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36923 } }
I20260808 10:44:50.881002 7484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:50.881038 7484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d40648583c04eb2bb98c171854d8439, State: Initialized, Role: FOLLOWER
I20260808 10:44:50.881394 7484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d40648583c04eb2bb98c171854d8439" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36923 } }
I20260808 10:44:50.881505 7484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:50.881548 7484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:50.881619 7484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:50.882421 7484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d40648583c04eb2bb98c171854d8439" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36923 } }
I20260808 10:44:50.882787 7484 leader_election.cc:302] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d40648583c04eb2bb98c171854d8439; no voters:
I20260808 10:44:50.883134 7484 leader_election.cc:288] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:50.883260 7491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:50.883544 7491 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [term 1 LEADER]: Becoming Leader. State: Replica: 8d40648583c04eb2bb98c171854d8439, State: Running, Role: LEADER
I20260808 10:44:50.883885 7491 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d40648583c04eb2bb98c171854d8439" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36923 } }
I20260808 10:44:50.884399 7484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:50.885562 7492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d40648583c04eb2bb98c171854d8439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d40648583c04eb2bb98c171854d8439" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36923 } } }
I20260808 10:44:50.885703 7492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:50.886008 7493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d40648583c04eb2bb98c171854d8439. Latest consensus state: current_term: 1 leader_uuid: "8d40648583c04eb2bb98c171854d8439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d40648583c04eb2bb98c171854d8439" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36923 } } }
I20260808 10:44:50.886090 7506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:50.886123 7493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:50.888840 7506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:50.893954 7506 catalog_manager.cc:1347] Generated new cluster ID: 7298e9a3611849d2afcd3658dabd1f9d
I20260808 10:44:50.894018 7506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:50.903110 7506 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:50.903446 7506 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:50.911689 7506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439: Generated new TSK 0
I20260808 10:44:50.912505 7506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:44:50.922276 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:36923
--builtin_ntp_servers=127.31.237.148:42983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:44:51.041395 7518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:51.041689 7518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:51.041761 7518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:51.045751 7518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:51.045828 7518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:51.045920 7518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:44:51.050675 7518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:36923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:51.051836 7518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:51.053357 7518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:51.063309 7529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:51.063285 7530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:51.063464 7518 server_base.cc:1034] running on GCE node
W20260808 10:44:51.063285 7533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:51.063974 7518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:51.064677 7518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:51.065841 7518 hybrid_clock.cc:648] HybridClock initialized: now 1786185891065816 us; error 38 us; skew 500 ppm
I20260808 10:44:51.067845 7518 webserver.cc:460] Webserver started at http://127.31.237.129:40843/ using document root <none> and password file <none>
I20260808 10:44:51.068456 7518 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:51.068523 7518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:51.068847 7518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:51.071326 7518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "5e92fd439b6b49a481b049f95ff22ba4"
format_stamp: "Formatted at 2026-08-08 10:44:51 on dist-test-slave-twwt"
server_key: "508f03196303547432f67dd6826142d1"
server_key_iv: "bf3ef6b9a63aafa99b214801586ac0e6"
server_key_version: "encryptionkey@0"
I20260808 10:44:51.071933 7518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "5e92fd439b6b49a481b049f95ff22ba4"
format_stamp: "Formatted at 2026-08-08 10:44:51 on dist-test-slave-twwt"
server_key: "508f03196303547432f67dd6826142d1"
server_key_iv: "bf3ef6b9a63aafa99b214801586ac0e6"
server_key_version: "encryptionkey@0"
I20260808 10:44:51.075723 7518 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:44:51.078261 7541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:51.079555 7518 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:44:51.079697 7518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "5e92fd439b6b49a481b049f95ff22ba4"
format_stamp: "Formatted at 2026-08-08 10:44:51 on dist-test-slave-twwt"
server_key: "508f03196303547432f67dd6826142d1"
server_key_iv: "bf3ef6b9a63aafa99b214801586ac0e6"
server_key_version: "encryptionkey@0"
I20260808 10:44:51.079820 7518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:51.084527 7518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:51.085274 7518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:51.085474 7518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:51.086114 7518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:51.087153 7518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:51.087225 7518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:51.087294 7518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:51.087349 7518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:51.096184 7518 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:45419
I20260808 10:44:51.096258 7714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:45419 every 8 connection(s)
I20260808 10:44:51.097123 7518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:44:51.099640 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 7518
I20260808 10:44:51.099800 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:44:51.100132 32694 external_mini_cluster.cc:1428] Setting key 7aa5293349297e5e18dc57fca84b68fb
I20260808 10:44:51.102394 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:36923
--builtin_ntp_servers=127.31.237.148:42983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 10:44:51.104267 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.099048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:46249 (local address 127.31.237.190:36923)
0808 10:44:51.099190 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:51.099194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:51.100449 (+ 1255us) server_negotiation.cc:407] Connection header received
0808 10:44:51.102308 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:51.102310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:51.102351 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:51.102413 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:51.103690 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:51.103693 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:51.103695 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:51.103707 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:51.103726 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:51.103734 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:51.103809 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:51.104032 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 10:44:51.104077 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:44:51.105104 7718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.099447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36923 (local address 127.31.237.129:46249)
0808 10:44:51.100270 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:51.100326 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:51.102060 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:51.102637 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:51.102649 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:51.102764 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:51.103370 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:44:51.103374 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:51.103435 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:44:51.103531 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:51.103827 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:51.103839 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:51.103848 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:44:51.104037 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 10:44:51.104318 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":121,"threads_started":1}
I20260808 10:44:51.107035 7715 heartbeater.cc:347] Connected to a master server at 127.31.237.190:36923
I20260808 10:44:51.107287 7715 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:51.107699 7715 heartbeater.cc:511] Master 127.31.237.190:36923 requested a full tablet report, sending...
I20260808 10:44:51.109325 7430 ts_manager.cc:194] Registered new tserver with Master: 5e92fd439b6b49a481b049f95ff22ba4 (127.31.237.129:45419)
W20260808 10:44:51.222679 7719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:51.222995 7719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:51.223073 7719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:51.227077 7719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:51.227169 7719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:51.227262 7719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:44:51.232179 7719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:36923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:51.233362 7719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:51.234843 7719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:51.244750 7730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:51.244763 7727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:51.244764 7732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:51.244838 7719 server_base.cc:1034] running on GCE node
I20260808 10:44:51.245385 7719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:51.246075 7719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:51.247264 7719 hybrid_clock.cc:648] HybridClock initialized: now 1786185891247238 us; error 45 us; skew 500 ppm
I20260808 10:44:51.249063 7719 webserver.cc:460] Webserver started at http://127.31.237.130:43183/ using document root <none> and password file <none>
I20260808 10:44:51.249668 7719 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:51.249765 7719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:51.250016 7719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:51.252375 7719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "92aeaa3c1d984d82944dcc04099646bd"
format_stamp: "Formatted at 2026-08-08 10:44:51 on dist-test-slave-twwt"
server_key: "f7c50c91db9e9c6ae0b002f86d802336"
server_key_iv: "4103a2c88bfedbcad4804b39ceb926c8"
server_key_version: "encryptionkey@0"
I20260808 10:44:51.252997 7719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "92aeaa3c1d984d82944dcc04099646bd"
format_stamp: "Formatted at 2026-08-08 10:44:51 on dist-test-slave-twwt"
server_key: "f7c50c91db9e9c6ae0b002f86d802336"
server_key_iv: "4103a2c88bfedbcad4804b39ceb926c8"
server_key_version: "encryptionkey@0"
I20260808 10:44:51.256858 7719 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:44:51.259462 7739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:51.260784 7719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:44:51.260882 7719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "92aeaa3c1d984d82944dcc04099646bd"
format_stamp: "Formatted at 2026-08-08 10:44:51 on dist-test-slave-twwt"
server_key: "f7c50c91db9e9c6ae0b002f86d802336"
server_key_iv: "4103a2c88bfedbcad4804b39ceb926c8"
server_key_version: "encryptionkey@0"
I20260808 10:44:51.261037 7719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:51.265686 7719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:51.266295 7719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:51.266480 7719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:51.267073 7719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:51.267988 7719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:51.268057 7719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:51.268116 7719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:51.268141 7719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:51.278815 7719 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:44883
I20260808 10:44:51.278898 7917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:44883 every 8 connection(s)
I20260808 10:44:51.279789 7719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:44:51.281320 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 7719
I20260808 10:44:51.281452 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:44:51.281792 32694 external_mini_cluster.cc:1428] Setting key ddef26bbf1b4b640ca9a28d247aa091c
I20260808 10:44:51.284162 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:36923
--builtin_ntp_servers=127.31.237.148:42983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 10:44:51.287328 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.282170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:42551 (local address 127.31.237.190:36923)
0808 10:44:51.282345 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:51.282351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:51.283661 (+ 1310us) server_negotiation.cc:407] Connection header received
0808 10:44:51.285295 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:51.285299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:51.285344 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:51.285448 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:51.286828 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:51.286832 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:51.286835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:51.286848 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:51.286871 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:51.286883 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:51.286992 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:51.287148 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 10:44:51.287195 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:44:51.288174 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.282705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36923 (local address 127.31.237.130:42551)
0808 10:44:51.283523 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:51.283568 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:51.285038 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:51.285702 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:51.285714 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:51.285833 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:51.286515 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:44:51.286516 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:51.286557 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:44:51.286681 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:51.286989 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:51.286999 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:51.287007 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:44:51.287194 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 10:44:51.287448 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":133,"threads_started":1}
I20260808 10:44:51.289368 7918 heartbeater.cc:347] Connected to a master server at 127.31.237.190:36923
I20260808 10:44:51.289639 7918 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:51.290165 7918 heartbeater.cc:511] Master 127.31.237.190:36923 requested a full tablet report, sending...
I20260808 10:44:51.291437 7430 ts_manager.cc:194] Registered new tserver with Master: 92aeaa3c1d984d82944dcc04099646bd (127.31.237.130:44883)
W20260808 10:44:51.402945 7927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:51.403185 7927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:51.403254 7927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:51.407244 7927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:51.407305 7927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:51.407394 7927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:44:51.412210 7927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:36923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:51.413378 7927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:51.414899 7927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:51.424624 7938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:51.424708 7935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:51.424624 7934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:51.424822 7927 server_base.cc:1034] running on GCE node
I20260808 10:44:51.425348 7927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:51.426036 7927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:51.427222 7927 hybrid_clock.cc:648] HybridClock initialized: now 1786185891427211 us; error 32 us; skew 500 ppm
I20260808 10:44:51.429083 7927 webserver.cc:460] Webserver started at http://127.31.237.131:36613/ using document root <none> and password file <none>
I20260808 10:44:51.429672 7927 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:51.429766 7927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:51.430016 7927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:51.432268 7927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "d987332b3254407ba4e84807c76dff7f"
format_stamp: "Formatted at 2026-08-08 10:44:51 on dist-test-slave-twwt"
server_key: "c3dcaa76c9406005c62213cfe44e47f1"
server_key_iv: "afb7bfe3be480234bcdbb8b716469839"
server_key_version: "encryptionkey@0"
I20260808 10:44:51.432857 7927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "d987332b3254407ba4e84807c76dff7f"
format_stamp: "Formatted at 2026-08-08 10:44:51 on dist-test-slave-twwt"
server_key: "c3dcaa76c9406005c62213cfe44e47f1"
server_key_iv: "afb7bfe3be480234bcdbb8b716469839"
server_key_version: "encryptionkey@0"
I20260808 10:44:51.436673 7927 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:44:51.439239 7947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:51.440526 7927 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:44:51.440657 7927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "d987332b3254407ba4e84807c76dff7f"
format_stamp: "Formatted at 2026-08-08 10:44:51 on dist-test-slave-twwt"
server_key: "c3dcaa76c9406005c62213cfe44e47f1"
server_key_iv: "afb7bfe3be480234bcdbb8b716469839"
server_key_version: "encryptionkey@0"
I20260808 10:44:51.440775 7927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:51.445346 7927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:51.445950 7927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:51.446138 7927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:51.446762 7927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:51.447697 7927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:51.447774 7927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:51.447847 7927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:51.447906 7927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:51.456880 7927 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:37679
I20260808 10:44:51.456959 8128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:37679 every 8 connection(s)
I20260808 10:44:51.457841 7927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:44:51.461874 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 7927
I20260808 10:44:51.462004 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:44:51.462306 32694 external_mini_cluster.cc:1428] Setting key e9f6805ce36a4a2fec0839e5ce646ddb
I20260808 10:44:51.464870 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.459944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:45189 (local address 127.31.237.190:36923)
0808 10:44:51.460107 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:51.460110 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:51.461219 (+ 1109us) server_negotiation.cc:407] Connection header received
0808 10:44:51.462850 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:51.462854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:51.462902 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:51.463003 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:51.464331 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:51.464335 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:51.464337 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:51.464350 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:51.464370 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:51.464378 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:51.464484 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:51.464661 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 10:44:51.464708 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:44:51.465761 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.460325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36923 (local address 127.31.237.131:45189)
0808 10:44:51.461059 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:51.461100 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:51.462542 (+ 1442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:51.463248 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:51.463258 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:51.463383 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:51.464032 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:44:51.464034 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:51.464090 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:44:51.464180 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:51.464493 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:51.464503 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:51.464512 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:44:51.464692 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 10:44:51.465033 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":130,"threads_started":1}
I20260808 10:44:51.466921 8129 heartbeater.cc:347] Connected to a master server at 127.31.237.190:36923
I20260808 10:44:51.467166 8129 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:51.467590 8129 heartbeater.cc:511] Master 127.31.237.190:36923 requested a full tablet report, sending...
I20260808 10:44:51.468619 7430 ts_manager.cc:194] Registered new tserver with Master: d987332b3254407ba4e84807c76dff7f (127.31.237.131:37679)
I20260808 10:44:51.476702 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:44:51.480248 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.478783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47698 (local address 127.31.237.190:36923)
0808 10:44:51.478943 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:51.478949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:51.479088 (+ 139us) server_negotiation.cc:407] Connection header received
0808 10:44:51.479199 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:51.479201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:51.479240 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:51.479306 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:51.479894 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:51.479898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:51.479900 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:51.479912 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:51.479931 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:51.479938 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:51.480005 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:51.480045 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 10:44:51.480077 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:44:51.483115 7430 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:44:51.486517 7430 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:44:51.498921 8146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.495541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:37679 (local address 127.0.0.1:58436)
0808 10:44:51.496197 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:51.496211 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:51.496270 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:51.497264 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:51.497270 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:51.497346 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:51.498048 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:44:51.498052 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:51.498078 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:44:51.498157 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:51.498507 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:51.498515 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:51.498521 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:44:51.498679 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 10:44:51.498755 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":109,"threads_started":1}
I20260808 10:44:51.498936 8147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.495342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:44883 (local address 127.0.0.1:34482)
0808 10:44:51.495896 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:51.495959 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:51.496066 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:51.497382 (+ 1316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:51.497387 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:51.497403 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:51.498289 (+ 886us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:44:51.498291 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:51.498301 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:44:51.498321 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:51.498752 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:51.498755 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:51.498758 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:44:51.498825 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 10:44:51.498853 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":87,"threads_started":1}
I20260808 10:44:51.499251 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.495545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34482 (local address 127.31.237.130:44883)
0808 10:44:51.496015 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:51.496028 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:51.496067 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:44:51.496239 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:51.496249 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:51.496474 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:51.496620 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:51.498467 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:51.498472 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:51.498486 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:51.498501 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:51.498549 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:51.498589 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:51.498711 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:51.499003 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 10:44:51.499083 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":110,"threads_started":1}
I20260808 10:44:51.499336 8151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.496075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:45419 (local address 127.0.0.1:60210)
0808 10:44:51.496461 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:51.496473 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:51.496542 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:51.497912 (+ 1370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:51.497917 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:51.497939 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:51.498335 (+ 396us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:44:51.498337 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:51.498358 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:44:51.498387 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:51.498865 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:51.498868 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:51.498871 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:44:51.498945 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 10:44:51.498984 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":95,"threads_started":1}
I20260808 10:44:51.498972 8152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.496352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58436 (local address 127.31.237.131:37679)
0808 10:44:51.496791 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:51.496804 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:51.496831 (+ 27us) server_negotiation.cc:407] Connection header received
0808 10:44:51.496899 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:51.496905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:51.497084 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:51.497662 (+ 578us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:51.498328 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:51.498334 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:51.498348 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:51.498360 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:51.498400 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:51.498433 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:51.498540 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:51.498707 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 10:44:51.498795 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":102,"threads_started":1}
I20260808 10:44:51.500690 8153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.496371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60210 (local address 127.31.237.129:45419)
0808 10:44:51.496911 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:51.496922 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:51.496942 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:44:51.496985 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:51.496989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:51.497118 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:51.497254 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:51.498541 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:51.498548 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:51.498562 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:51.498576 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:51.498664 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:51.498704 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:51.498820 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:51.499135 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 10:44:51.499207 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":94,"threads_started":1}
I20260808 10:44:51.502291 7818 tablet_service.cc:1467] Processing CreateTablet for tablet 9da0bb5d4fdb4be683eb3907694e5541 (DEFAULT_TABLE table=test-table [id=8bbe9d496c9949369ed74eca622bff85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:51.502631 8022 tablet_service.cc:1467] Processing CreateTablet for tablet 9da0bb5d4fdb4be683eb3907694e5541 (DEFAULT_TABLE table=test-table [id=8bbe9d496c9949369ed74eca622bff85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:51.503525 7818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da0bb5d4fdb4be683eb3907694e5541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:51.503692 8022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da0bb5d4fdb4be683eb3907694e5541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:51.503327 7619 tablet_service.cc:1467] Processing CreateTablet for tablet 9da0bb5d4fdb4be683eb3907694e5541 (DEFAULT_TABLE table=test-table [id=8bbe9d496c9949369ed74eca622bff85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:44:51.504311 7619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da0bb5d4fdb4be683eb3907694e5541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:51.509996 8155 tablet_bootstrap.cc:492] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd: Bootstrap starting.
I20260808 10:44:51.511709 8159 tablet_bootstrap.cc:492] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4: Bootstrap starting.
I20260808 10:44:51.512166 8156 tablet_bootstrap.cc:492] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f: Bootstrap starting.
I20260808 10:44:51.512550 8155 tablet_bootstrap.cc:654] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:51.513314 8155 log.cc:824] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:51.514040 8156 tablet_bootstrap.cc:654] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:51.514851 8156 log.cc:824] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:51.515363 8155 tablet_bootstrap.cc:492] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd: No bootstrap required, opened a new log
I20260808 10:44:51.515583 8155 ts_tablet_manager.cc:1397] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260808 10:44:51.516762 8159 tablet_bootstrap.cc:654] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:51.516891 8156 tablet_bootstrap.cc:492] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f: No bootstrap required, opened a new log
I20260808 10:44:51.517084 8156 ts_tablet_manager.cc:1397] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260808 10:44:51.517613 8159 log.cc:824] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:51.519706 8159 tablet_bootstrap.cc:492] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4: No bootstrap required, opened a new log
I20260808 10:44:51.519896 8159 ts_tablet_manager.cc:1397] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 10:44:51.519819 8155 raft_consensus.cc:348] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aeaa3c1d984d82944dcc04099646bd" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 44883 } } peers { permanent_uuid: "d987332b3254407ba4e84807c76dff7f" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 37679 } } peers { permanent_uuid: "5e92fd439b6b49a481b049f95ff22ba4" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45419 } }
I20260808 10:44:51.520112 8155 raft_consensus.cc:374] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:51.520200 8155 raft_consensus.cc:729] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92aeaa3c1d984d82944dcc04099646bd, State: Initialized, Role: FOLLOWER
I20260808 10:44:51.520673 8155 consensus_queue.cc:260] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aeaa3c1d984d82944dcc04099646bd" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 44883 } } peers { permanent_uuid: "d987332b3254407ba4e84807c76dff7f" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 37679 } } peers { permanent_uuid: "5e92fd439b6b49a481b049f95ff22ba4" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45419 } }
I20260808 10:44:51.521155 8156 raft_consensus.cc:348] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aeaa3c1d984d82944dcc04099646bd" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 44883 } } peers { permanent_uuid: "d987332b3254407ba4e84807c76dff7f" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 37679 } } peers { permanent_uuid: "5e92fd439b6b49a481b049f95ff22ba4" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45419 } }
I20260808 10:44:51.521390 7918 heartbeater.cc:503] Master 127.31.237.190:36923 was elected leader, sending a full tablet report...
I20260808 10:44:51.521445 8156 raft_consensus.cc:374] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:51.521512 8156 raft_consensus.cc:729] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d987332b3254407ba4e84807c76dff7f, State: Initialized, Role: FOLLOWER
I20260808 10:44:51.521929 8156 consensus_queue.cc:260] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aeaa3c1d984d82944dcc04099646bd" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 44883 } } peers { permanent_uuid: "d987332b3254407ba4e84807c76dff7f" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 37679 } } peers { permanent_uuid: "5e92fd439b6b49a481b049f95ff22ba4" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45419 } }
I20260808 10:44:51.522583 8129 heartbeater.cc:503] Master 127.31.237.190:36923 was elected leader, sending a full tablet report...
I20260808 10:44:51.522961 8156 ts_tablet_manager.cc:1428] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260808 10:44:51.524163 8155 ts_tablet_manager.cc:1428] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260808 10:44:51.524572 8159 raft_consensus.cc:348] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aeaa3c1d984d82944dcc04099646bd" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 44883 } } peers { permanent_uuid: "d987332b3254407ba4e84807c76dff7f" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 37679 } } peers { permanent_uuid: "5e92fd439b6b49a481b049f95ff22ba4" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45419 } }
I20260808 10:44:51.524791 8159 raft_consensus.cc:374] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:51.524873 8159 raft_consensus.cc:729] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e92fd439b6b49a481b049f95ff22ba4, State: Initialized, Role: FOLLOWER
I20260808 10:44:51.525278 8159 consensus_queue.cc:260] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aeaa3c1d984d82944dcc04099646bd" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 44883 } } peers { permanent_uuid: "d987332b3254407ba4e84807c76dff7f" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 37679 } } peers { permanent_uuid: "5e92fd439b6b49a481b049f95ff22ba4" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45419 } }
I20260808 10:44:51.525977 7715 heartbeater.cc:503] Master 127.31.237.190:36923 was elected leader, sending a full tablet report...
I20260808 10:44:51.526270 8159 ts_tablet_manager.cc:1428] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260808 10:44:51.530788 7922 tablet.cc:2369] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:44:51.598465 7716 tablet.cc:2369] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:44:51.708926 8130 tablet.cc:2369] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:44:51.906777 8167 raft_consensus.cc:482] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:44:51.907068 8167 raft_consensus.cc:504] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aeaa3c1d984d82944dcc04099646bd" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 44883 } } peers { permanent_uuid: "d987332b3254407ba4e84807c76dff7f" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 37679 } } peers { permanent_uuid: "5e92fd439b6b49a481b049f95ff22ba4" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45419 } }
I20260808 10:44:51.908360 8167 leader_election.cc:288] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d987332b3254407ba4e84807c76dff7f (127.31.237.131:37679), 5e92fd439b6b49a481b049f95ff22ba4 (127.31.237.129:45419)
I20260808 10:44:51.910728 8152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.908761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:41359 (local address 127.31.237.131:37679)
0808 10:44:51.908935 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:51.908940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:51.909088 (+ 148us) server_negotiation.cc:407] Connection header received
0808 10:44:51.909274 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:51.909278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:51.909324 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:51.909384 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:51.910282 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:51.910285 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:51.910287 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:51.910298 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:51.910315 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:51.910323 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:51.910421 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:51.910531 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 10:44:51.910571 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 10:44:51.910756 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.908615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:37679 (local address 127.31.237.130:41359)
0808 10:44:51.908976 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:51.908995 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:51.909138 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:51.909523 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:51.909526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:51.909542 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:51.910088 (+ 546us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:44:51.910089 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:51.910100 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:44:51.910126 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:51.910418 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:51.910421 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:51.910425 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:44:51.910529 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 10:44:51.910571 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":102,"threads_started":1}
I20260808 10:44:51.910928 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.909086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:45419 (local address 127.31.237.130:60645)
0808 10:44:51.909338 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:51.909350 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:51.909428 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:51.909659 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:51.909661 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:51.909673 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:51.910265 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:44:51.910267 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:51.910276 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:44:51.910302 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:51.910598 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:51.910601 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:51.910636 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 10:44:51.910741 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 10:44:51.910779 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":66,"threads_started":1}
I20260808 10:44:51.910915 8153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.909244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:60645 (local address 127.31.237.129:45419)
0808 10:44:51.909404 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:51.909409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:51.909425 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:44:51.909514 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:51.909517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:51.909551 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:51.909631 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:51.910445 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:51.910449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:51.910452 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:51.910465 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:51.910489 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:51.910503 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:51.910601 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:51.910746 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 10:44:51.910788 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 10:44:51.911653 7645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da0bb5d4fdb4be683eb3907694e5541" candidate_uuid: "92aeaa3c1d984d82944dcc04099646bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e92fd439b6b49a481b049f95ff22ba4" is_pre_election: true
I20260808 10:44:51.911659 8049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da0bb5d4fdb4be683eb3907694e5541" candidate_uuid: "92aeaa3c1d984d82944dcc04099646bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d987332b3254407ba4e84807c76dff7f" is_pre_election: true
I20260808 10:44:51.911979 7645 raft_consensus.cc:2454] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92aeaa3c1d984d82944dcc04099646bd in term 0.
I20260808 10:44:51.911990 8049 raft_consensus.cc:2454] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92aeaa3c1d984d82944dcc04099646bd in term 0.
I20260808 10:44:51.912446 7744 leader_election.cc:302] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e92fd439b6b49a481b049f95ff22ba4, 92aeaa3c1d984d82944dcc04099646bd; no voters:
I20260808 10:44:51.912751 8167 raft_consensus.cc:2781] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:44:51.912829 8167 raft_consensus.cc:482] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:44:51.912860 8167 raft_consensus.cc:3037] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:51.913959 8167 raft_consensus.cc:504] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aeaa3c1d984d82944dcc04099646bd" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 44883 } } peers { permanent_uuid: "d987332b3254407ba4e84807c76dff7f" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 37679 } } peers { permanent_uuid: "5e92fd439b6b49a481b049f95ff22ba4" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45419 } }
I20260808 10:44:51.914305 8167 leader_election.cc:288] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [CANDIDATE]: Term 1 election: Requested vote from peers d987332b3254407ba4e84807c76dff7f (127.31.237.131:37679), 5e92fd439b6b49a481b049f95ff22ba4 (127.31.237.129:45419)
I20260808 10:44:51.914736 8049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da0bb5d4fdb4be683eb3907694e5541" candidate_uuid: "92aeaa3c1d984d82944dcc04099646bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d987332b3254407ba4e84807c76dff7f"
I20260808 10:44:51.914798 7645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da0bb5d4fdb4be683eb3907694e5541" candidate_uuid: "92aeaa3c1d984d82944dcc04099646bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e92fd439b6b49a481b049f95ff22ba4"
I20260808 10:44:51.914880 8049 raft_consensus.cc:3037] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:51.914943 7645 raft_consensus.cc:3037] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:51.915980 8049 raft_consensus.cc:2454] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92aeaa3c1d984d82944dcc04099646bd in term 1.
I20260808 10:44:51.915992 7645 raft_consensus.cc:2454] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92aeaa3c1d984d82944dcc04099646bd in term 1.
I20260808 10:44:51.916257 7744 leader_election.cc:302] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92aeaa3c1d984d82944dcc04099646bd, d987332b3254407ba4e84807c76dff7f; no voters:
I20260808 10:44:51.916447 8167 raft_consensus.cc:2781] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:51.916672 8167 raft_consensus.cc:686] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [term 1 LEADER]: Becoming Leader. State: Replica: 92aeaa3c1d984d82944dcc04099646bd, State: Running, Role: LEADER
I20260808 10:44:51.917048 8167 consensus_queue.cc:237] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aeaa3c1d984d82944dcc04099646bd" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 44883 } } peers { permanent_uuid: "d987332b3254407ba4e84807c76dff7f" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 37679 } } peers { permanent_uuid: "5e92fd439b6b49a481b049f95ff22ba4" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45419 } }
I20260808 10:44:51.920469 7428 catalog_manager.cc:5495] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 92aeaa3c1d984d82944dcc04099646bd (127.31.237.130). New cstate: current_term: 1 leader_uuid: "92aeaa3c1d984d82944dcc04099646bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92aeaa3c1d984d82944dcc04099646bd" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 44883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d987332b3254407ba4e84807c76dff7f" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 37679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e92fd439b6b49a481b049f95ff22ba4" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45419 } health_report { overall_health: UNKNOWN } } }
I20260808 10:44:51.953392 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:51.951500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34486 (local address 127.31.237.130:44883)
0808 10:44:51.951884 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:51.951888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:51.951901 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:44:51.951981 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:51.951984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:51.952064 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:51.952163 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:51.952962 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:51.952966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:51.952968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:51.952982 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:51.953010 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:51.953019 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:51.953115 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:51.953196 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 10:44:51.953240 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263}
I20260808 10:44:51.961151 8049 raft_consensus.cc:1261] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f [term 1 FOLLOWER]: Refusing update from remote peer 92aeaa3c1d984d82944dcc04099646bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:51.961210 7645 raft_consensus.cc:1261] T 9da0bb5d4fdb4be683eb3907694e5541 P 5e92fd439b6b49a481b049f95ff22ba4 [term 1 FOLLOWER]: Refusing update from remote peer 92aeaa3c1d984d82944dcc04099646bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:44:51.961988 8174 consensus_queue.cc:1035] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d987332b3254407ba4e84807c76dff7f" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 37679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:51.962237 8167 consensus_queue.cc:1035] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e92fd439b6b49a481b049f95ff22ba4" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:44:51.971724 8183 mvcc.cc:205] Tried to move back new op lower bound from 7316217413464797184 to 7316217413295198208. Current Snapshot: MvccSnapshot[applied={T|T < 7316217413464797184}]
I20260808 10:44:51.978065 7429 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:51.978274 7429 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:44:51.980871 7429 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d40648583c04eb2bb98c171854d8439: Sending DeleteTablet for 3 replicas of tablet 9da0bb5d4fdb4be683eb3907694e5541
I20260808 10:44:51.981714 7818 tablet_service.cc:1514] Processing DeleteTablet for tablet 9da0bb5d4fdb4be683eb3907694e5541 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:51 UTC) from {username='slave'} at 127.0.0.1:34482
I20260808 10:44:51.981887 7619 tablet_service.cc:1514] Processing DeleteTablet for tablet 9da0bb5d4fdb4be683eb3907694e5541 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:51 UTC) from {username='slave'} at 127.0.0.1:60210
I20260808 10:44:51.982278 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 7518
I20260808 10:44:51.982478 8022 tablet_service.cc:1514] Processing DeleteTablet for tablet 9da0bb5d4fdb4be683eb3907694e5541 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:44:51 UTC) from {username='slave'} at 127.0.0.1:58436
I20260808 10:44:51.988497 8199 tablet_replica.cc:330] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f: stopping tablet replica
I20260808 10:44:51.988931 8199 raft_consensus.cc:2229] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:51.989341 8199 raft_consensus.cc:2258] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:51.989768 8197 tablet_replica.cc:330] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd: stopping tablet replica
I20260808 10:44:51.991364 8199 ts_tablet_manager.cc:1905] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:51.994285 8199 ts_tablet_manager.cc:1918] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:44:51.994398 8199 log.cc:1196] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/9da0bb5d4fdb4be683eb3907694e5541
I20260808 10:44:51.994771 8197 raft_consensus.cc:2229] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:44:51.995162 8197 raft_consensus.cc:2258] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:51.995693 8199 ts_tablet_manager.cc:1939] T 9da0bb5d4fdb4be683eb3907694e5541 P d987332b3254407ba4e84807c76dff7f: Deleting consensus metadata
I20260808 10:44:51.996814 8197 ts_tablet_manager.cc:1905] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:51.997098 7407 catalog_manager.cc:4841] TS d987332b3254407ba4e84807c76dff7f (127.31.237.131:37679): tablet 9da0bb5d4fdb4be683eb3907694e5541 (table test-table [id=8bbe9d496c9949369ed74eca622bff85]) successfully deleted
I20260808 10:44:52.000339 8197 ts_tablet_manager.cc:1918] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:44:52.000476 8197 log.cc:1196] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/9da0bb5d4fdb4be683eb3907694e5541
I20260808 10:44:52.000842 8197 ts_tablet_manager.cc:1939] T 9da0bb5d4fdb4be683eb3907694e5541 P 92aeaa3c1d984d82944dcc04099646bd: Deleting consensus metadata
I20260808 10:44:52.002228 7404 catalog_manager.cc:4841] TS 92aeaa3c1d984d82944dcc04099646bd (127.31.237.130:44883): tablet 9da0bb5d4fdb4be683eb3907694e5541 (table test-table [id=8bbe9d496c9949369ed74eca622bff85]) successfully deleted
W20260808 10:44:52.003420 7407 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:45419 (error 108)
I20260808 10:44:52.003548 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 7719
I20260808 10:44:52.013569 8201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:52.013089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:45419 (local address 127.0.0.1:60226)
0808 10:44:52.013359 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:52.013455 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:45419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":66,"threads_started":1}
W20260808 10:44:52.013803 7407 catalog_manager.cc:4584] TS 5e92fd439b6b49a481b049f95ff22ba4 (127.31.237.129:45419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9da0bb5d4fdb4be683eb3907694e5541: Network error: Client connection negotiation failed: client connection to 127.31.237.129:45419: connect: Connection refused (error 111)
I20260808 10:44:52.019119 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 7927
I20260808 10:44:52.031816 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 7384
2026-08-08T10:44:52Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1394 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 10:44:52.057451 32694 mini_oidc.cc:152] Starting JWKS server
I20260808 10:44:52.059293 32694 webserver.cc:460] Webserver started at https://127.0.0.1:37575/ using document root <none> and password file <none>
I20260808 10:44:52.059499 32694 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:44:52.059788 32694 webserver.cc:460] Webserver started at http://127.0.0.1:35459/ using document root <none> and password file <none>
2026-08-08T10:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:52Z Disabled control of system clock
I20260808 10:44:52.070590 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:37101
--webserver_interface=127.31.237.190
--webserver_port=0
--builtin_ntp_servers=127.31.237.148:35829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:37101
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37575/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/testchainca.cert with env {}
W20260808 10:44:52.184165 8210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:52.184438 8210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:52.184478 8210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:52.188601 8210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:52.188673 8210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:52.188688 8210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:52.188706 8210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:52.188721 8210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:44:52.188786 8210 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:44:52.188809 8210 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37575/jwks.json
I20260808 10:44:52.194341 8210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:35829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:37101
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:37101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37575/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:52.195480 8210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:52.196548 8210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:52.205274 8221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:52.205315 8217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:52.205297 8218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:52.205673 8210 server_base.cc:1034] running on GCE node
I20260808 10:44:52.206240 8210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:52.207376 8210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:52.208559 8210 hybrid_clock.cc:648] HybridClock initialized: now 1786185892208536 us; error 31 us; skew 500 ppm
I20260808 10:44:52.210374 8210 webserver.cc:460] Webserver started at http://127.31.237.190:46019/ using document root <none> and password file <none>
I20260808 10:44:52.211015 8210 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:52.211081 8210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:52.211252 8210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:52.213301 8210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "2a8667c5e4aa4c0a99884de2152407aa"
format_stamp: "Formatted at 2026-08-08 10:44:52 on dist-test-slave-twwt"
server_key: "ec7e78c80b9ecd3205a8eab58a0abafc"
server_key_iv: "d1cade35043670915c4f7c50e579ac2c"
server_key_version: "encryptionkey@0"
I20260808 10:44:52.213833 8210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "2a8667c5e4aa4c0a99884de2152407aa"
format_stamp: "Formatted at 2026-08-08 10:44:52 on dist-test-slave-twwt"
server_key: "ec7e78c80b9ecd3205a8eab58a0abafc"
server_key_iv: "d1cade35043670915c4f7c50e579ac2c"
server_key_version: "encryptionkey@0"
I20260808 10:44:52.217518 8210 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 10:44:52.219959 8229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:52.221187 8210 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:44:52.221292 8210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "2a8667c5e4aa4c0a99884de2152407aa"
format_stamp: "Formatted at 2026-08-08 10:44:52 on dist-test-slave-twwt"
server_key: "ec7e78c80b9ecd3205a8eab58a0abafc"
server_key_iv: "d1cade35043670915c4f7c50e579ac2c"
server_key_version: "encryptionkey@0"
I20260808 10:44:52.221375 8210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:52.235392 8210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:52.236039 8210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:52.236193 8210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:52.243577 8210 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:37101
I20260808 10:44:52.243654 8308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:37101 every 8 connection(s)
I20260808 10:44:52.244612 8210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:52.246783 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 8210
I20260808 10:44:52.246888 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:52.247208 32694 external_mini_cluster.cc:1428] Setting key c65452e221b4e7182f82c09fa02090d6
I20260808 10:44:52.248643 8309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:52.256381 8309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa: Bootstrap starting.
I20260808 10:44:52.258921 8313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:52.248577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42482 (local address 127.31.237.190:37101)
0808 10:44:52.249328 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:52.249350 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:52.249396 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:44:52.250350 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:52.250375 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:52.250931 (+ 556us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:52.251342 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:52.251829 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:52.253517 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:52.254439 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:52.255291 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:52.255470 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:52.256356 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:52.256363 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:52.256380 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:52.256926 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:52.256998 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:52.257029 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:52.257179 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:52.257387 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 10:44:52.257870 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":157,"threads_started":1}
I20260808 10:44:52.259416 8309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:52.260355 8309 log.cc:824] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:52.262001 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:44:52.262594 8309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa: No bootstrap required, opened a new log
I20260808 10:44:52.266240 8309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8667c5e4aa4c0a99884de2152407aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 37101 } }
I20260808 10:44:52.266463 8309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:52.266500 8309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a8667c5e4aa4c0a99884de2152407aa, State: Initialized, Role: FOLLOWER
I20260808 10:44:52.266990 8309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8667c5e4aa4c0a99884de2152407aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 37101 } }
I20260808 10:44:52.267134 8309 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:52.267202 8309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:52.267328 8309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:52.268225 8309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8667c5e4aa4c0a99884de2152407aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 37101 } }
I20260808 10:44:52.268565 8309 leader_election.cc:302] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a8667c5e4aa4c0a99884de2152407aa; no voters:
I20260808 10:44:52.268869 8309 leader_election.cc:288] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:52.269024 8317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:52.269295 8317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [term 1 LEADER]: Becoming Leader. State: Replica: 2a8667c5e4aa4c0a99884de2152407aa, State: Running, Role: LEADER
I20260808 10:44:52.269667 8317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8667c5e4aa4c0a99884de2152407aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 37101 } }
I20260808 10:44:52.270176 8309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:52.271430 8319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a8667c5e4aa4c0a99884de2152407aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8667c5e4aa4c0a99884de2152407aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 37101 } } }
I20260808 10:44:52.271559 8319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:52.271910 8320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a8667c5e4aa4c0a99884de2152407aa. Latest consensus state: current_term: 1 leader_uuid: "2a8667c5e4aa4c0a99884de2152407aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8667c5e4aa4c0a99884de2152407aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 37101 } } }
I20260808 10:44:52.271975 8329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:52.272042 8320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:52.275198 8329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:52.281118 8329 catalog_manager.cc:1347] Generated new cluster ID: ce0e62173a60485ead6b93d3f5a2a43c
I20260808 10:44:52.281177 8329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:52.309397 8329 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:52.310815 8329 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:52.322418 8329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a8667c5e4aa4c0a99884de2152407aa: Generated new TSK 0
I20260808 10:44:52.323194 8329 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 10:44:52.360069 8357 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 10:44:52.360235 8357 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 10:44:52.360834 8313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:52.336439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48332 (local address 127.31.237.190:37101)
0808 10:44:52.336575 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:52.336579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:52.336591 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:44:52.336665 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:52.336668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:52.336715 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:52.336834 (+ 119us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:44:52.337111 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:52.337632 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:52.338592 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:52.338890 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:52.339010 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:52.339400 (+ 390us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:44:52.360283 (+ 20883us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 10:44:52.360656 (+ 373us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48332: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 10:44:52.360941 8313 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48332: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 10:44:52.362140 8356 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:52.335769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:37101 (local address 127.0.0.1:48332)
0808 10:44:52.336113 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:52.336126 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:52.336234 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:52.336863 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:52.336867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:52.336876 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:44:52.336991 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:52.336996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:52.337770 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:52.337774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:52.338461 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:52.338470 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:52.339022 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:52.339026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:52.339120 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:52.339173 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:44:52.360576 (+ 21403us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 10:44:52.360884 (+ 308us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.237.190:37101: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":123,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.31.237.190:37101: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 10:44:52.362958 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 8210
2026-08-08T10:44:52Z chronyd exiting
I20260808 10:44:52.410699 32694 test_util.cc:175] -----------------------------------------------
I20260808 10:44:52.410794 32694 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786185862616645-32694-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (355 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 10:44:52.411978 32694 mini_oidc.cc:152] Starting JWKS server
I20260808 10:44:52.412909 32694 webserver.cc:460] Webserver started at https://127.0.0.1:36741/ using document root <none> and password file <none>
I20260808 10:44:52.412998 32694 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:44:52.413338 32694 webserver.cc:460] Webserver started at http://127.0.0.1:41753/ using document root <none> and password file <none>
2026-08-08T10:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:52Z Disabled control of system clock
I20260808 10:44:52.423848 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:36893
--webserver_interface=127.31.237.190
--webserver_port=0
--builtin_ntp_servers=127.31.237.148:32969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:36893
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36741/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/testchainca.cert with env {}
W20260808 10:44:52.539379 8365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:52.539674 8365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:52.539778 8365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:52.543859 8365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:52.543941 8365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:52.543968 8365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:52.544008 8365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:52.544050 8365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:44:52.544104 8365 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:44:52.544137 8365 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36741/jwks.json
I20260808 10:44:52.550132 8365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:32969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:36893
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:36893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36741/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:52.551378 8365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:52.552513 8365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:52.561954 8376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:52.562036 8372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:52.562089 8365 server_base.cc:1034] running on GCE node
W20260808 10:44:52.561949 8374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:52.562985 8365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:52.564098 8365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:52.565281 8365 hybrid_clock.cc:648] HybridClock initialized: now 1786185892565264 us; error 30 us; skew 500 ppm
I20260808 10:44:52.567445 8365 webserver.cc:460] Webserver started at http://127.31.237.190:39233/ using document root <none> and password file <none>
I20260808 10:44:52.568113 8365 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:52.568181 8365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:52.568454 8365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:52.570663 8365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "9de15a2a4bee45f7bf9c4c38e44cbb53"
format_stamp: "Formatted at 2026-08-08 10:44:52 on dist-test-slave-twwt"
server_key: "9c366551067dd467bebca7abf4a974ab"
server_key_iv: "f151a8081031656b4986d07450669798"
server_key_version: "encryptionkey@0"
I20260808 10:44:52.571281 8365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "9de15a2a4bee45f7bf9c4c38e44cbb53"
format_stamp: "Formatted at 2026-08-08 10:44:52 on dist-test-slave-twwt"
server_key: "9c366551067dd467bebca7abf4a974ab"
server_key_iv: "f151a8081031656b4986d07450669798"
server_key_version: "encryptionkey@0"
I20260808 10:44:52.576965 8365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 10:44:52.579684 8388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:52.581058 8365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:52.581311 8365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "9de15a2a4bee45f7bf9c4c38e44cbb53"
format_stamp: "Formatted at 2026-08-08 10:44:52 on dist-test-slave-twwt"
server_key: "9c366551067dd467bebca7abf4a974ab"
server_key_iv: "f151a8081031656b4986d07450669798"
server_key_version: "encryptionkey@0"
I20260808 10:44:52.581440 8365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:52.591004 8365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:52.591707 8365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:52.591926 8365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:52.598829 8365 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:36893
I20260808 10:44:52.598834 8468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:36893 every 8 connection(s)
I20260808 10:44:52.599835 8365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:52.603446 8469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:52.609494 8469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53: Bootstrap starting.
I20260808 10:44:52.610097 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 8365
I20260808 10:44:52.610248 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:52.610540 32694 external_mini_cluster.cc:1428] Setting key b61c4f7b2c57fe4d94968d81de835e81
I20260808 10:44:52.612401 8469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:52.613271 8469 log.cc:824] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:52.615588 8469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53: No bootstrap required, opened a new log
I20260808 10:44:52.620071 8469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de15a2a4bee45f7bf9c4c38e44cbb53" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36893 } }
I20260808 10:44:52.620292 8469 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:52.620352 8469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9de15a2a4bee45f7bf9c4c38e44cbb53, State: Initialized, Role: FOLLOWER
I20260808 10:44:52.620582 8474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:52.611989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47684 (local address 127.31.237.190:36893)
0808 10:44:52.612752 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:52.612772 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:52.612819 (+ 47us) server_negotiation.cc:407] Connection header received
0808 10:44:52.613683 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:52.613711 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:52.614152 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:52.614485 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:52.614917 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:52.615929 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:52.617139 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:52.617619 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:52.617736 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:52.618679 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:52.618684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:52.618697 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:52.619100 (+ 403us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:52.619157 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:52.619177 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:52.619261 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:52.619447 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 10:44:52.619744 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":148,"threads_started":1}
I20260808 10:44:52.620862 8469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de15a2a4bee45f7bf9c4c38e44cbb53" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36893 } }
I20260808 10:44:52.621002 8469 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:52.621089 8469 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:52.621215 8469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:52.622334 8469 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de15a2a4bee45f7bf9c4c38e44cbb53" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36893 } }
I20260808 10:44:52.622766 8469 leader_election.cc:302] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9de15a2a4bee45f7bf9c4c38e44cbb53; no voters:
I20260808 10:44:52.623154 8469 leader_election.cc:288] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:52.623258 8476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:52.623498 8476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [term 1 LEADER]: Becoming Leader. State: Replica: 9de15a2a4bee45f7bf9c4c38e44cbb53, State: Running, Role: LEADER
I20260808 10:44:52.623845 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:44:52.624435 8469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:52.624694 8476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de15a2a4bee45f7bf9c4c38e44cbb53" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36893 } }
I20260808 10:44:52.626529 8477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9de15a2a4bee45f7bf9c4c38e44cbb53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de15a2a4bee45f7bf9c4c38e44cbb53" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36893 } } }
I20260808 10:44:52.626689 8477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:52.627043 8478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9de15a2a4bee45f7bf9c4c38e44cbb53. Latest consensus state: current_term: 1 leader_uuid: "9de15a2a4bee45f7bf9c4c38e44cbb53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de15a2a4bee45f7bf9c4c38e44cbb53" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36893 } } }
I20260808 10:44:52.627128 8478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53 [sys.catalog]: This master's current role is: LEADER
W20260808 10:44:52.629587 8492 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:52.629666 8492 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:52.629741 8495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:52.630600 8495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:52.640167 8495 catalog_manager.cc:1347] Generated new cluster ID: 53396d959795480f8ef06947623305a3
I20260808 10:44:52.640259 8495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:52.649013 8495 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:52.650238 8495 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:52.663084 8495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9de15a2a4bee45f7bf9c4c38e44cbb53: Generated new TSK 0
I20260808 10:44:52.663892 8495 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 10:44:52.686340 8512 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 10:44:52.686501 8512 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 10:44:52.687238 8474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:52.671232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47692 (local address 127.31.237.190:36893)
0808 10:44:52.671357 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:52.671361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:52.671623 (+ 262us) server_negotiation.cc:407] Connection header received
0808 10:44:52.671769 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:52.671773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:52.671834 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:52.671950 (+ 116us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:44:52.672160 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:52.673007 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:52.673859 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:52.674236 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:52.674353 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:52.674656 (+ 303us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:44:52.686778 (+ 12122us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36741/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 10:44:52.687080 (+ 302us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47692: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36741/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 10:44:52.687314 8474 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47692: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36741/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 10:44:52.687326 8511 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:52.671161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36893 (local address 127.0.0.1:47692)
0808 10:44:52.671524 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:52.671537 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:52.671638 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:52.671951 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:52.671953 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:52.671959 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:44:52.672058 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:52.672062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:52.673121 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:52.673123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:52.673750 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:52.673758 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:52.674360 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:52.674362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:52.674440 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:52.674463 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:44:52.686982 (+ 12519us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36741/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 10:44:52.687152 (+ 170us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.237.190:36893: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36741/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":113,"threads_started":1}
I20260808 10:44:52.688076 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 8365
2026-08-08T10:44:52Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (324 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 10:44:52.736429 32694 mini_oidc.cc:152] Starting JWKS server
I20260808 10:44:52.737309 32694 webserver.cc:460] Webserver started at https://127.0.0.1:46683/ using document root <none> and password file <none>
I20260808 10:44:52.737430 32694 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:44:52.737720 32694 webserver.cc:460] Webserver started at http://127.0.0.1:44937/ using document root <none> and password file <none>
2026-08-08T10:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:52Z Disabled control of system clock
I20260808 10:44:52.748183 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:39611
--webserver_interface=127.31.237.190
--webserver_port=0
--builtin_ntp_servers=127.31.237.148:44877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:39611
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46683/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 10:44:52.865522 8519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:52.865797 8519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:52.865836 8519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:52.869813 8519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:52.869874 8519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:52.869889 8519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:52.869906 8519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:52.869922 8519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:44:52.869937 8519 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:44:52.869951 8519 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46683/jwks.json
I20260808 10:44:52.874747 8519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:39611
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:39611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46683/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:52.875782 8519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:52.877127 8519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:52.886004 8532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:52.885964 8530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:52.886040 8536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:52.886189 8519 server_base.cc:1034] running on GCE node
I20260808 10:44:52.886943 8519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:52.888082 8519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:52.889259 8519 hybrid_clock.cc:648] HybridClock initialized: now 1786185892889240 us; error 37 us; skew 500 ppm
I20260808 10:44:52.891317 8519 webserver.cc:460] Webserver started at http://127.31.237.190:44829/ using document root <none> and password file <none>
I20260808 10:44:52.891922 8519 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:52.891986 8519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:52.892208 8519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:52.894330 8519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "bb4a79f36169449590851d09ce58d034"
format_stamp: "Formatted at 2026-08-08 10:44:52 on dist-test-slave-twwt"
server_key: "f831ff2afca57ec9ce570ca4cb14a29d"
server_key_iv: "8961fdae9249a7be6030e6e600f27740"
server_key_version: "encryptionkey@0"
I20260808 10:44:52.894970 8519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "bb4a79f36169449590851d09ce58d034"
format_stamp: "Formatted at 2026-08-08 10:44:52 on dist-test-slave-twwt"
server_key: "f831ff2afca57ec9ce570ca4cb14a29d"
server_key_iv: "8961fdae9249a7be6030e6e600f27740"
server_key_version: "encryptionkey@0"
I20260808 10:44:52.898958 8519 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:44:52.901616 8546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:52.903124 8519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:52.903313 8519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "bb4a79f36169449590851d09ce58d034"
format_stamp: "Formatted at 2026-08-08 10:44:52 on dist-test-slave-twwt"
server_key: "f831ff2afca57ec9ce570ca4cb14a29d"
server_key_iv: "8961fdae9249a7be6030e6e600f27740"
server_key_version: "encryptionkey@0"
I20260808 10:44:52.903456 8519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:52.920644 8519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:52.921443 8519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:52.921681 8519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:52.929993 8519 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:39611
I20260808 10:44:52.929992 8619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:39611 every 8 connection(s)
I20260808 10:44:52.931177 8519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:52.934510 8623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:52.934785 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 8519
I20260808 10:44:52.934952 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:52.935246 32694 external_mini_cluster.cc:1428] Setting key d21bd500d68f54e3e47d268ee13e88b7
I20260808 10:44:52.942759 8623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034: Bootstrap starting.
I20260808 10:44:52.945803 8623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:52.946848 8628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:52.936724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52474 (local address 127.31.237.190:39611)
0808 10:44:52.937482 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:52.937503 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:52.937553 (+ 50us) server_negotiation.cc:407] Connection header received
0808 10:44:52.938404 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:52.938433 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:52.939065 (+ 632us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:52.939486 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:52.939988 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:52.941659 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:52.942639 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:52.943399 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:52.943569 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:52.944463 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:52.944470 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:52.944487 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:52.945034 (+ 547us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:52.945113 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:52.945145 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:52.945248 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:52.945400 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 10:44:52.945712 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":116,"threads_started":1}
I20260808 10:44:52.946825 8623 log.cc:824] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:52.949520 8623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034: No bootstrap required, opened a new log
I20260808 10:44:52.950345 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:44:52.955864 8623 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4a79f36169449590851d09ce58d034" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39611 } }
I20260808 10:44:52.956229 8623 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:52.956315 8623 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4a79f36169449590851d09ce58d034, State: Initialized, Role: FOLLOWER
I20260808 10:44:52.956902 8623 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4a79f36169449590851d09ce58d034" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39611 } }
I20260808 10:44:52.957108 8623 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:52.957185 8623 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:52.957321 8623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:52.959089 8623 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4a79f36169449590851d09ce58d034" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39611 } }
I20260808 10:44:52.959617 8623 leader_election.cc:302] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb4a79f36169449590851d09ce58d034; no voters:
I20260808 10:44:52.960117 8623 leader_election.cc:288] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:52.960201 8632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:52.960423 8632 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [term 1 LEADER]: Becoming Leader. State: Replica: bb4a79f36169449590851d09ce58d034, State: Running, Role: LEADER
I20260808 10:44:52.960759 8632 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4a79f36169449590851d09ce58d034" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39611 } }
I20260808 10:44:52.961612 8623 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:52.962564 8634 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb4a79f36169449590851d09ce58d034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4a79f36169449590851d09ce58d034" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39611 } } }
I20260808 10:44:52.962718 8634 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:52.962913 8633 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb4a79f36169449590851d09ce58d034. Latest consensus state: current_term: 1 leader_uuid: "bb4a79f36169449590851d09ce58d034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4a79f36169449590851d09ce58d034" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39611 } } }
I20260808 10:44:52.963248 8633 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:52.963517 8643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:52.967253 8643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:52.977916 8643 catalog_manager.cc:1347] Generated new cluster ID: 62c5a5923f304ae5aafb185331ef200e
I20260808 10:44:52.978015 8643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:52.990347 8643 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:52.991346 8643 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:53.002411 8643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb4a79f36169449590851d09ce58d034: Generated new TSK 0
I20260808 10:44:53.003217 8643 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:44:53.023428 8628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:53.017645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52478 (local address 127.31.237.190:39611)
0808 10:44:53.017794 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:53.017799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:53.017971 (+ 172us) server_negotiation.cc:407] Connection header received
0808 10:44:53.018112 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:53.018115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:53.018174 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:53.018298 (+ 124us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:44:53.018587 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:53.019471 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:53.020414 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:53.020778 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:53.020884 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:53.021351 (+ 467us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:44:53.023015 (+ 1664us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46683/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 10:44:53.023276 (+ 261us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52478: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46683/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 10:44:53.023511 8628 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52478: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46683/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 10:44:53.026168 8666 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:53.017476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39611 (local address 127.0.0.1:52478)
0808 10:44:53.017882 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:53.017895 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:53.017997 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:53.018334 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:53.018337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:53.018361 (+ 24us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:44:53.018466 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:53.018470 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:53.019608 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:53.019611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:53.020294 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:53.020302 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:53.020928 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:53.020932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:53.021004 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:53.021038 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:44:53.023692 (+ 2654us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46683/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 10:44:53.025976 (+ 2284us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.237.190:39611: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46683/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":2036,"thread_start_us":132,"threads_started":1}
I20260808 10:44:53.026965 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 8519
E20260808 10:44:53.045148 8669 webserver.cc:533] Webserver: sslize failed:
E20260808 10:44:53.045266 8669 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T10:44:53Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (341 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T10:44:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:53Z Disabled control of system clock
I20260808 10:44:53.088511 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:45983
--webserver_interface=127.31.237.190
--webserver_port=0
--builtin_ntp_servers=127.31.237.148:33153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:45983
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/testchainca.cert with env {}
W20260808 10:44:53.204061 8674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:53.204387 8674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:53.204484 8674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:53.208581 8674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:53.208699 8674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:53.208729 8674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:53.208770 8674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:53.208806 8674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:44:53.208843 8674 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:44:53.208873 8674 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 10:44:53.214507 8674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:33153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:45983
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:45983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:53.215724 8674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:53.216843 8674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:53.226034 8688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:53.225993 8683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:53.226166 8674 server_base.cc:1034] running on GCE node
W20260808 10:44:53.225977 8684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:53.226949 8674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:53.228133 8674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:53.229331 8674 hybrid_clock.cc:648] HybridClock initialized: now 1786185893229313 us; error 34 us; skew 500 ppm
I20260808 10:44:53.231395 8674 webserver.cc:460] Webserver started at http://127.31.237.190:42175/ using document root <none> and password file <none>
I20260808 10:44:53.232007 8674 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:53.232103 8674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:53.232343 8674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:53.234485 8674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "265c720a0f3e4525a643c8076062cc76"
format_stamp: "Formatted at 2026-08-08 10:44:53 on dist-test-slave-twwt"
server_key: "ad58e0dd92899c828dd529b1e5b3781c"
server_key_iv: "eea0d96c1a84a0a821b69743c2ce13b8"
server_key_version: "encryptionkey@0"
I20260808 10:44:53.235150 8674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "265c720a0f3e4525a643c8076062cc76"
format_stamp: "Formatted at 2026-08-08 10:44:53 on dist-test-slave-twwt"
server_key: "ad58e0dd92899c828dd529b1e5b3781c"
server_key_iv: "eea0d96c1a84a0a821b69743c2ce13b8"
server_key_version: "encryptionkey@0"
I20260808 10:44:53.238989 8674 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:44:53.241496 8696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:53.242820 8674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:44:53.242992 8674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "265c720a0f3e4525a643c8076062cc76"
format_stamp: "Formatted at 2026-08-08 10:44:53 on dist-test-slave-twwt"
server_key: "ad58e0dd92899c828dd529b1e5b3781c"
server_key_iv: "eea0d96c1a84a0a821b69743c2ce13b8"
server_key_version: "encryptionkey@0"
I20260808 10:44:53.243117 8674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:53.260936 8674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:53.261656 8674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:53.261869 8674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:53.269134 8674 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:45983
I20260808 10:44:53.269147 8784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:45983 every 8 connection(s)
I20260808 10:44:53.270104 8674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:53.273001 8785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:53.275136 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 8674
I20260808 10:44:53.275290 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:53.275578 32694 external_mini_cluster.cc:1428] Setting key 8772caf7b8a3b6a8a7ff039bcf995236
I20260808 10:44:53.279660 8785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76: Bootstrap starting.
I20260808 10:44:53.282308 8785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:53.283281 8785 log.cc:824] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:53.285472 8785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76: No bootstrap required, opened a new log
I20260808 10:44:53.286047 8792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:53.277035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54354 (local address 127.31.237.190:45983)
0808 10:44:53.277750 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:53.277770 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:53.277813 (+ 43us) server_negotiation.cc:407] Connection header received
0808 10:44:53.278638 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:53.278662 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:53.279190 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:53.279573 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:53.280033 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:53.281509 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:53.282408 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:53.283144 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:53.283320 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:53.284222 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:53.284235 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:53.284249 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:44:53.284645 (+ 396us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:53.284702 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:44:53.284724 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:53.284848 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:53.285003 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 10:44:53.285272 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":95,"threads_started":1}
I20260808 10:44:53.289144 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:44:53.289747 8785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265c720a0f3e4525a643c8076062cc76" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45983 } }
I20260808 10:44:53.290009 8785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:53.290119 8785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265c720a0f3e4525a643c8076062cc76, State: Initialized, Role: FOLLOWER
I20260808 10:44:53.290584 8785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265c720a0f3e4525a643c8076062cc76" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45983 } }
I20260808 10:44:53.290812 8785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:53.290895 8785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:53.291009 8785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:53.292001 8785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265c720a0f3e4525a643c8076062cc76" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45983 } }
I20260808 10:44:53.292323 8785 leader_election.cc:302] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 265c720a0f3e4525a643c8076062cc76; no voters:
I20260808 10:44:53.292610 8785 leader_election.cc:288] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:53.292775 8796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:53.293026 8796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [term 1 LEADER]: Becoming Leader. State: Replica: 265c720a0f3e4525a643c8076062cc76, State: Running, Role: LEADER
I20260808 10:44:53.293417 8796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265c720a0f3e4525a643c8076062cc76" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45983 } }
I20260808 10:44:53.293926 8785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:53.295256 8798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 265c720a0f3e4525a643c8076062cc76. Latest consensus state: current_term: 1 leader_uuid: "265c720a0f3e4525a643c8076062cc76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265c720a0f3e4525a643c8076062cc76" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45983 } } }
I20260808 10:44:53.295390 8798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:53.295753 8797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "265c720a0f3e4525a643c8076062cc76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265c720a0f3e4525a643c8076062cc76" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45983 } } }
I20260808 10:44:53.296475 8797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76 [sys.catalog]: This master's current role is: LEADER
W20260808 10:44:53.299777 8819 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:53.299885 8819 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:53.299968 8809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:53.300761 8809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:53.307030 8809 catalog_manager.cc:1347] Generated new cluster ID: d2755f4a8c844df7919e82019c5b2632
I20260808 10:44:53.307104 8809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:53.317022 8809 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:53.317922 8809 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:53.324883 8809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 265c720a0f3e4525a643c8076062cc76: Generated new TSK 0
I20260808 10:44:53.325709 8809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:44:54.173627 8792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:53.343520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54364 (local address 127.31.237.190:45983)
0808 10:44:53.343674 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:53.343680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:53.343871 (+ 191us) server_negotiation.cc:407] Connection header received
0808 10:44:53.344057 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:53.344060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:53.344112 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:53.344244 (+ 132us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:44:53.344502 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:53.345054 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:53.345971 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:53.346204 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:53.346323 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:53.346776 (+ 453us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:44:54.173203 (+826427us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 10:44:54.173500 (+ 297us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54364: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 10:44:54.173753 8792 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54364: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 10:44:54.173902 8833 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:53.343434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:45983 (local address 127.0.0.1:54364)
0808 10:44:53.343753 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:53.343788 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:53.343916 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:53.344243 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:53.344247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:53.344255 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:44:53.344372 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:53.344379 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:53.345180 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:53.345184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:53.345828 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:53.345836 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:53.346396 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:53.346401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:53.346512 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:53.346544 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:44:54.173475 (+826931us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 10:44:54.173669 (+ 194us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.237.190:45983: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":110,"threads_started":1}
I20260808 10:44:54.174567 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 8674
2026-08-08T10:44:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1124 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (128 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (131 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:54 dist-test-slave-twwt krb5kdc[8852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:54 dist-test-slave-twwt krb5kdc[8852](info): set up 2 sockets
Aug 08 10:44:54 dist-test-slave-twwt krb5kdc[8852](info): commencing operation
krb5kdc: starting...
W20260808 10:44:55.706519 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:44:55 dist-test-slave-twwt krb5kdc[8852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:44:55.841523 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:40333
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:43489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:40333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:55.962821 8871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:55.963130 8871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:55.963209 8871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:55.967340 8871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:55.967425 8871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:55.967463 8871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:55.967501 8871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:55.967540 8871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:55.972913 8871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:43489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:40333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:40333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:55.973991 8871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:55.975107 8871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:55.983820 8884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:55.983808 8881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:55.983925 8880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:55.984170 8871 server_base.cc:1034] running on GCE node
I20260808 10:44:55.984827 8871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:55.985942 8871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:55.987115 8871 hybrid_clock.cc:648] HybridClock initialized: now 1786185895987078 us; error 49 us; skew 500 ppm
Aug 08 10:44:55 dist-test-slave-twwt krb5kdc[8852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185895, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:55.993556 8871 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:44:55.994534 8871 webserver.cc:460] Webserver started at http://127.31.237.190:41503/ using document root <none> and password file <none>
I20260808 10:44:55.995182 8871 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:55.995249 8871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:55.995507 8871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:55.997759 8871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "c315c0cbe9f249d3ba97db43d7f16c70"
format_stamp: "Formatted at 2026-08-08 10:44:55 on dist-test-slave-twwt"
server_key: "32eb8931c7fcb4357b9042a15f4ca807"
server_key_iv: "d8f6b931d8c8d92a38a9ad376ff24890"
server_key_version: "encryptionkey@0"
I20260808 10:44:55.998377 8871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "c315c0cbe9f249d3ba97db43d7f16c70"
format_stamp: "Formatted at 2026-08-08 10:44:55 on dist-test-slave-twwt"
server_key: "32eb8931c7fcb4357b9042a15f4ca807"
server_key_iv: "d8f6b931d8c8d92a38a9ad376ff24890"
server_key_version: "encryptionkey@0"
I20260808 10:44:56.002297 8871 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 10:44:56.004832 8898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:56.006168 8871 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:56.006333 8871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "c315c0cbe9f249d3ba97db43d7f16c70"
format_stamp: "Formatted at 2026-08-08 10:44:55 on dist-test-slave-twwt"
server_key: "32eb8931c7fcb4357b9042a15f4ca807"
server_key_iv: "d8f6b931d8c8d92a38a9ad376ff24890"
server_key_version: "encryptionkey@0"
I20260808 10:44:56.006459 8871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:56.025597 8871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:56.028864 8871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:56.029117 8871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:56.036798 8871 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:40333
I20260808 10:44:56.036814 8981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:40333 every 8 connection(s)
I20260808 10:44:56.037823 8871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:56.039543 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 8871
I20260808 10:44:56.039685 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:56.040061 32694 external_mini_cluster.cc:1428] Setting key 18c1a31bedd69e1f51ba688b7566822d
I20260808 10:44:56.041659 8986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:56.048328 8986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70: Bootstrap starting.
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:56.051026 8986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:56.051893 8986 log.cc:824] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:56.054214 8986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70: No bootstrap required, opened a new log
I20260808 10:44:56.055478 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:56.042017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49452 (local address 127.31.237.190:40333)
0808 10:44:56.042443 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:56.042466 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:56.042511 (+ 45us) server_negotiation.cc:407] Connection header received
0808 10:44:56.043264 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:56.043282 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:56.043608 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:56.043891 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:56.044872 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.045851 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.047061 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.047523 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.047645 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:56.050423 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:56.050429 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:56.050443 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:56.050471 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:56.052387 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:56.052897 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:56.052903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:56.052909 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:56.053044 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:56.053307 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:56.053310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:56.053312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:56.053825 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:56.054032 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:56.054215 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 10:44:56.054526 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":148,"threads_started":1}
I20260808 10:44:56.058022 8986 raft_consensus.cc:348] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c315c0cbe9f249d3ba97db43d7f16c70" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40333 } }
I20260808 10:44:56.058310 8986 raft_consensus.cc:374] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:56.058391 8986 raft_consensus.cc:729] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c315c0cbe9f249d3ba97db43d7f16c70, State: Initialized, Role: FOLLOWER
I20260808 10:44:56.058869 8986 consensus_queue.cc:260] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c315c0cbe9f249d3ba97db43d7f16c70" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40333 } }
I20260808 10:44:56.059019 8986 raft_consensus.cc:388] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:56.059105 8986 raft_consensus.cc:482] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:56.059243 8986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:56.060187 8986 raft_consensus.cc:504] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c315c0cbe9f249d3ba97db43d7f16c70" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40333 } }
I20260808 10:44:56.060554 8986 leader_election.cc:302] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c315c0cbe9f249d3ba97db43d7f16c70; no voters:
I20260808 10:44:56.060882 8986 leader_election.cc:288] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:56.061065 8996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:56.061302 8996 raft_consensus.cc:686] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [term 1 LEADER]: Becoming Leader. State: Replica: c315c0cbe9f249d3ba97db43d7f16c70, State: Running, Role: LEADER
I20260808 10:44:56.061691 8996 consensus_queue.cc:237] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c315c0cbe9f249d3ba97db43d7f16c70" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40333 } }
I20260808 10:44:56.062155 8986 sys_catalog.cc:564] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:56.063526 8997 sys_catalog.cc:455] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c315c0cbe9f249d3ba97db43d7f16c70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c315c0cbe9f249d3ba97db43d7f16c70" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40333 } } }
I20260808 10:44:56.063666 8997 sys_catalog.cc:458] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:56.063983 8998 sys_catalog.cc:455] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c315c0cbe9f249d3ba97db43d7f16c70. Latest consensus state: current_term: 1 leader_uuid: "c315c0cbe9f249d3ba97db43d7f16c70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c315c0cbe9f249d3ba97db43d7f16c70" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40333 } } }
I20260808 10:44:56.064082 8998 sys_catalog.cc:458] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70 [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:56.064899 9010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:56.067821 9010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:56.074258 9010 catalog_manager.cc:1347] Generated new cluster ID: ce073928a4a3452f85b46ba0cf467262
I20260808 10:44:56.074339 9010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:56.092530 9010 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:56.093540 9010 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:56.104960 9010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c315c0cbe9f249d3ba97db43d7f16c70: Generated new TSK 0
I20260808 10:44:56.105679 9010 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:44:56.139686 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40333
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:43489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:56.257395 9024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:56.257678 9024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:56.257722 9024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:56.261854 9024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:56.261950 9024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:56.262012 9024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:44:56.267815 9024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:43489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:56.268966 9024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:56.270100 9024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:56.280083 9032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:56.280074 9033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:56.280074 9037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:56.280160 9024 server_base.cc:1034] running on GCE node
I20260808 10:44:56.280781 9024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:56.281427 9024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:56.282665 9024 hybrid_clock.cc:648] HybridClock initialized: now 1786185896282645 us; error 29 us; skew 500 ppm
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:56.285426 9024 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:44:56.286332 9024 webserver.cc:460] Webserver started at http://127.31.237.129:44677/ using document root <none> and password file <none>
I20260808 10:44:56.286988 9024 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:56.287087 9024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:56.287328 9024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:56.289366 9024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "54050f3454c04e6e8b4c4c8dfcb44771"
format_stamp: "Formatted at 2026-08-08 10:44:56 on dist-test-slave-twwt"
server_key: "66a0822060c5a2ad487f75b4d9f91cc3"
server_key_iv: "fd34a3cd58f4f689e3909f5b454b98e7"
server_key_version: "encryptionkey@0"
I20260808 10:44:56.289948 9024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "54050f3454c04e6e8b4c4c8dfcb44771"
format_stamp: "Formatted at 2026-08-08 10:44:56 on dist-test-slave-twwt"
server_key: "66a0822060c5a2ad487f75b4d9f91cc3"
server_key_iv: "fd34a3cd58f4f689e3909f5b454b98e7"
server_key_version: "encryptionkey@0"
I20260808 10:44:56.293754 9024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:44:56.296242 9045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:56.297473 9024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:44:56.297614 9024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "54050f3454c04e6e8b4c4c8dfcb44771"
format_stamp: "Formatted at 2026-08-08 10:44:56 on dist-test-slave-twwt"
server_key: "66a0822060c5a2ad487f75b4d9f91cc3"
server_key_iv: "fd34a3cd58f4f689e3909f5b454b98e7"
server_key_version: "encryptionkey@0"
I20260808 10:44:56.297727 9024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:56.320211 9024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:56.323328 9024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:56.323546 9024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:56.324563 9024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:56.325628 9024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:56.325706 9024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:56.325778 9024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:56.325809 9024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:56.335526 9024 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:39047
I20260808 10:44:56.335548 9221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:39047 every 8 connection(s)
I20260808 10:44:56.336452 9024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:44:56.337268 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 9024
I20260808 10:44:56.337412 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:44:56.337721 32694 external_mini_cluster.cc:1428] Setting key 4c8aa80a4aef888762555f9ef3d336e9
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
I20260808 10:44:56.352381 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:56.338531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:45999 (local address 127.31.237.190:40333)
0808 10:44:56.338718 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:56.338723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:56.339608 (+ 885us) server_negotiation.cc:407] Connection header received
0808 10:44:56.341723 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:56.341727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:56.341786 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:56.341869 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:56.343294 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.344167 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.345464 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.345934 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.346064 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:56.348642 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:56.348648 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:56.348651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:56.348684 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:56.350106 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:56.350654 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:56.350660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:56.350664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:56.350774 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:56.351131 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:56.351134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:56.351135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:56.351411 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:56.351521 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:56.352137 (+ 616us) server_negotiation.cc:299] Negotiation successful
0808 10:44:56.352253 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:44:56.353691 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:56.338872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:40333 (local address 127.31.237.129:45999)
0808 10:44:56.339440 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:56.339499 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:56.341457 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:56.342063 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:56.342071 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:56.342445 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:56.343015 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:56.343030 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.344304 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.344307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:56.345305 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:56.345316 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.346099 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.346101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:56.346244 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:56.346965 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:56.346983 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:56.348451 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:56.350230 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:56.350235 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:56.350248 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:56.350514 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:56.350890 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:56.350893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:56.350894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:56.351038 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:56.351535 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:56.351541 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:56.351829 (+ 288us) client_negotiation.cc:769] Sending connection context
0808 10:44:56.352067 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 10:44:56.352466 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":159,"threads_started":1}
I20260808 10:44:56.355083 9222 heartbeater.cc:347] Connected to a master server at 127.31.237.190:40333
I20260808 10:44:56.355360 9222 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:56.356011 9222 heartbeater.cc:511] Master 127.31.237.190:40333 requested a full tablet report, sending...
I20260808 10:44:56.357743 8918 ts_manager.cc:194] Registered new tserver with Master: 54050f3454c04e6e8b4c4c8dfcb44771 (127.31.237.129:39047)
I20260808 10:44:56.359069 8918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:45999
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:44:56.369979 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40333
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:43489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:56.485541 9229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:56.485800 9229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:56.485844 9229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:56.489856 9229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:56.489910 9229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:56.489969 9229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:44:56.495142 9229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:43489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:56.496155 9229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:56.497241 9229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:56.507360 9245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:56.507337 9240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:56.507364 9237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:56.507860 9229 server_base.cc:1034] running on GCE node
I20260808 10:44:56.508278 9229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:56.508956 9229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:56.510159 9229 hybrid_clock.cc:648] HybridClock initialized: now 1786185896510143 us; error 31 us; skew 500 ppm
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:56.513264 9229 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:44:56.514215 9229 webserver.cc:460] Webserver started at http://127.31.237.130:35781/ using document root <none> and password file <none>
I20260808 10:44:56.514880 9229 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:56.514977 9229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:56.515221 9229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:56.517541 9229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "88e4ddb7a77a4eac8b305e62fdb8605e"
format_stamp: "Formatted at 2026-08-08 10:44:56 on dist-test-slave-twwt"
server_key: "086aa07ea4c8e96ebfeb2136e9eb4431"
server_key_iv: "1a0e4467c67b4bee886f9359d7e6d478"
server_key_version: "encryptionkey@0"
I20260808 10:44:56.518165 9229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "88e4ddb7a77a4eac8b305e62fdb8605e"
format_stamp: "Formatted at 2026-08-08 10:44:56 on dist-test-slave-twwt"
server_key: "086aa07ea4c8e96ebfeb2136e9eb4431"
server_key_iv: "1a0e4467c67b4bee886f9359d7e6d478"
server_key_version: "encryptionkey@0"
I20260808 10:44:56.522042 9229 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:44:56.524578 9255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:56.525830 9229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:56.525966 9229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "88e4ddb7a77a4eac8b305e62fdb8605e"
format_stamp: "Formatted at 2026-08-08 10:44:56 on dist-test-slave-twwt"
server_key: "086aa07ea4c8e96ebfeb2136e9eb4431"
server_key_iv: "1a0e4467c67b4bee886f9359d7e6d478"
server_key_version: "encryptionkey@0"
I20260808 10:44:56.526079 9229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:56.556469 9229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:56.559988 9229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:56.560226 9229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:56.560904 9229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:56.561843 9229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:56.561920 9229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:56.561992 9229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:56.562044 9229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:56.572477 9229 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:34685
I20260808 10:44:56.572499 9435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:34685 every 8 connection(s)
I20260808 10:44:56.573477 9229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:44:56.577730 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 9229
I20260808 10:44:56.577875 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:44:56.578174 32694 external_mini_cluster.cc:1428] Setting key 22408a548ee2c34495c10b1cc3c16e1b
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:56.589298 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:56.575576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:60947 (local address 127.31.237.190:40333)
0808 10:44:56.575728 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:56.575732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:56.576745 (+ 1013us) server_negotiation.cc:407] Connection header received
0808 10:44:56.578304 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:56.578308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:56.578370 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:56.578490 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:56.579991 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.580523 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.581713 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.581948 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.582058 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:56.584673 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:56.584680 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:56.584688 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:56.584736 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:56.586721 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:56.587510 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:56.587513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:56.587514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:56.587595 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:56.588032 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:56.588035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:56.588036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:56.588222 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:56.588301 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:56.589057 (+ 756us) server_negotiation.cc:299] Negotiation successful
0808 10:44:56.589168 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:44:56.590425 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:56.575998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:40333 (local address 127.31.237.130:60947)
0808 10:44:56.576487 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:56.576532 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:56.578084 (+ 1552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:56.578761 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:56.578773 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:56.579235 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:56.579843 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:56.579858 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.580679 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.580682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:56.581582 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:56.581591 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.582119 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.582123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:56.582300 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:56.582989 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:56.583007 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:56.584490 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:56.586866 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:56.586877 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:56.586898 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:56.587383 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:56.587705 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:56.587709 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:56.587712 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:56.587927 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:56.588340 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:56.588351 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:56.588756 (+ 405us) client_negotiation.cc:769] Sending connection context
0808 10:44:56.589019 (+ 263us) client_negotiation.cc:240] Negotiation successful
0808 10:44:56.589439 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":124,"threads_started":1}
I20260808 10:44:56.591831 9438 heartbeater.cc:347] Connected to a master server at 127.31.237.190:40333
I20260808 10:44:56.592125 9438 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
I20260808 10:44:56.592794 9438 heartbeater.cc:511] Master 127.31.237.190:40333 requested a full tablet report, sending...
I20260808 10:44:56.594002 8918 ts_manager.cc:194] Registered new tserver with Master: 88e4ddb7a77a4eac8b305e62fdb8605e (127.31.237.130:34685)
I20260808 10:44:56.594949 8918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:60947
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:44:56.611615 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40333
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:43489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:56.729174 9445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:56.729430 9445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:56.729472 9445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:56.733512 9445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:56.733567 9445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:56.733623 9445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:44:56.738750 9445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:43489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:56.739712 9445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:56.740679 9445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:56.750424 9453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:56.750586 9445 server_base.cc:1034] running on GCE node
W20260808 10:44:56.750486 9456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:56.750422 9454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:56.751205 9445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:56.751871 9445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:56.753051 9445 hybrid_clock.cc:648] HybridClock initialized: now 1786185896753012 us; error 50 us; skew 500 ppm
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:56.756068 9445 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:44:56.757099 9445 webserver.cc:460] Webserver started at http://127.31.237.131:34965/ using document root <none> and password file <none>
I20260808 10:44:56.757717 9445 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:56.757815 9445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:56.758090 9445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:56.760475 9445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "f3d73bbb3a924411bfc096fe6ec1e39a"
format_stamp: "Formatted at 2026-08-08 10:44:56 on dist-test-slave-twwt"
server_key: "d40bb698d3215ac09e49009ac4e66ac2"
server_key_iv: "93cb2cfd5908de819e2392494822ecad"
server_key_version: "encryptionkey@0"
I20260808 10:44:56.761129 9445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "f3d73bbb3a924411bfc096fe6ec1e39a"
format_stamp: "Formatted at 2026-08-08 10:44:56 on dist-test-slave-twwt"
server_key: "d40bb698d3215ac09e49009ac4e66ac2"
server_key_iv: "93cb2cfd5908de819e2392494822ecad"
server_key_version: "encryptionkey@0"
I20260808 10:44:56.765555 9445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:44:56.768257 9467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:56.769609 9445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:44:56.769767 9445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "f3d73bbb3a924411bfc096fe6ec1e39a"
format_stamp: "Formatted at 2026-08-08 10:44:56 on dist-test-slave-twwt"
server_key: "d40bb698d3215ac09e49009ac4e66ac2"
server_key_iv: "93cb2cfd5908de819e2392494822ecad"
server_key_version: "encryptionkey@0"
I20260808 10:44:56.769905 9445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:56.783828 9445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:56.786755 9445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:56.786962 9445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:56.787571 9445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:56.788439 9445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:56.788512 9445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:56.788579 9445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:56.788635 9445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:56.798061 9445 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:44393
I20260808 10:44:56.798086 9638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:44393 every 8 connection(s)
I20260808 10:44:56.799204 9445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:44:56.799546 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 9445
I20260808 10:44:56.799695 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:44:56.800072 32694 external_mini_cluster.cc:1428] Setting key fe219cb2f90b70eab4632ab0eecc40e8
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:44:56.814563 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:56.801381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:47979 (local address 127.31.237.190:40333)
0808 10:44:56.801511 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:56.801515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:56.802716 (+ 1201us) server_negotiation.cc:407] Connection header received
0808 10:44:56.804157 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:56.804159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:56.804224 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:56.804315 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:56.805527 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.806053 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.807325 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.807548 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.807634 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:56.810362 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:56.810366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:56.810368 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:56.810394 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:56.812033 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:56.812535 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:56.812538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:56.812539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:56.812617 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:56.813185 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:56.813187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:56.813188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:56.813448 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:56.813553 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:56.814298 (+ 745us) server_negotiation.cc:299] Negotiation successful
0808 10:44:56.814439 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:44:56.815622 9642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:56.801939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:40333 (local address 127.31.237.131:47979)
0808 10:44:56.802555 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:56.802630 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:56.803939 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:56.804492 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:56.804500 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:56.804865 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:56.805383 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:56.805397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.806167 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.806170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:56.807207 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:56.807218 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.807696 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.807698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:56.807850 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:56.808560 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:56.808578 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:56.810144 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:56.812144 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:56.812150 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:56.812165 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:56.812444 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:56.812699 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:56.812701 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:56.812702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:56.812862 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:56.813589 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:56.813596 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:56.813989 (+ 393us) client_negotiation.cc:769] Sending connection context
0808 10:44:56.814221 (+ 232us) client_negotiation.cc:240] Negotiation successful
0808 10:44:56.814612 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":145,"threads_started":1}
I20260808 10:44:56.816869 9640 heartbeater.cc:347] Connected to a master server at 127.31.237.190:40333
I20260808 10:44:56.817142 9640 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:56.817664 9640 heartbeater.cc:511] Master 127.31.237.190:40333 requested a full tablet report, sending...
I20260808 10:44:56.818828 8918 ts_manager.cc:194] Registered new tserver with Master: f3d73bbb3a924411bfc096fe6ec1e39a (127.31.237.131:44393)
I20260808 10:44:56.819379 8918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:47979
I20260808 10:44:56.826834 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/test-admin.keytab.
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:56.843123 32694 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 10:44:56.846087 9650 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:56.844541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49300 (local address 127.31.237.129:39047)
0808 10:44:56.844882 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:56.844902 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:56.844947 (+ 45us) server_negotiation.cc:407] Connection header received
0808 10:44:56.845013 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:56.845019 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:56.845215 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:56.845368 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:56.845867 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49300: BlockingRecv error: recv got EOF from 127.0.0.1:49300 (error 108)
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":125,"threads_started":1}
W20260808 10:44:56.846194 32694 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:56.847313 32694 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:56.848268 32694 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.129@KRBTEST.COM
I20260808 10:44:56.858119 9650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:56.849147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49306 (local address 127.31.237.129:39047)
0808 10:44:56.849273 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:56.849278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:56.849384 (+ 106us) server_negotiation.cc:407] Connection header received
0808 10:44:56.849556 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:56.849559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:56.849600 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:56.849685 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:56.850225 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.851073 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.851987 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.852356 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.852481 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:56.854788 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:56.854804 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:56.854819 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:56.854843 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:56.856263 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:56.856721 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:56.856732 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:56.856733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:56.856809 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:56.857117 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:56.857121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:56.857122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:56.857382 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:56.857626 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:56.857809 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 10:44:56.857977 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 10:44:56.862059 9650 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:56.861078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49318 (local address 127.31.237.129:39047)
0808 10:44:56.861217 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:56.861221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:56.861248 (+ 27us) server_negotiation.cc:407] Connection header received
0808 10:44:56.861348 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:56.861350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:56.861389 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:56.861451 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:56.861927 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49318: BlockingRecv error: recv got EOF from 127.0.0.1:49318 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 10:44:56.862160 32694 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:56.863399 32694 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:56.864415 32694 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.129@KRBTEST.COM
I20260808 10:44:56.873381 9650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:56.865303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49320 (local address 127.31.237.129:39047)
0808 10:44:56.865436 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:56.865441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:56.865517 (+ 76us) server_negotiation.cc:407] Connection header received
0808 10:44:56.865672 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:56.865675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:56.865721 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:56.865801 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:56.866323 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.866910 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.867704 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.867938 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.868051 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:56.870302 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:56.870306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:56.870308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:56.870332 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:56.871647 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:56.872104 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:56.872108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:56.872109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:56.872188 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:56.872473 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:56.872477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:56.872479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:56.872657 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:56.872845 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:56.873089 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 10:44:56.873216 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 10:44:56.876529 9650 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:44:56.875373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49334 (local address 127.31.237.129:39047)
0808 10:44:56.875506 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:56.875510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:56.875576 (+ 66us) server_negotiation.cc:407] Connection header received
0808 10:44:56.875738 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:56.875741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:56.875778 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:56.875858 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:56.876420 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49334: BlockingRecv error: recv got EOF from 127.0.0.1:49334 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 10:44:56.876704 32694 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786185862616645-32694-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:56.877746 32694 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:56.878717 32694 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[8852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.129@KRBTEST.COM
I20260808 10:44:56.887789 9650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:56.879563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49336 (local address 127.31.237.129:39047)
0808 10:44:56.879682 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:56.879688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:56.879812 (+ 124us) server_negotiation.cc:407] Connection header received
0808 10:44:56.879910 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:56.879913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:56.879951 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:56.880008 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:56.880711 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.881513 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.882315 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:56.882545 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:56.882675 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:56.884923 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:56.884927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:56.884929 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:56.884952 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:56.886397 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:56.886814 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:56.886817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:56.886818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:56.886889 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:56.887129 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:56.887132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:56.887133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:56.887286 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:56.887392 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:56.887575 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 10:44:56.887668 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:44:56.889029 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 9024
I20260808 10:44:56.903878 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 9229
I20260808 10:44:56.914978 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 9445
I20260808 10:44:56.928686 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 8871
2026-08-08T10:44:56Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2495 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[9669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[9669](info): set up 2 sockets
Aug 08 10:44:56 dist-test-slave-twwt krb5kdc[9669](info): commencing operation
krb5kdc: starting...
W20260808 10:44:58.171972 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:44:58 dist-test-slave-twwt krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:44:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:44:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.190.keytab.
Entry for principal oryx/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.190.keytab.
I20260808 10:44:58.306046 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:36401
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.190.keytab
--principal=oryx/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:36401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:58.422591 9692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:58.422979 9692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:58.423076 9692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:58.427166 9692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:58.427273 9692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:58.427309 9692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:58.427346 9692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:58.427380 9692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:58.432786 9692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:36401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.190.keytab
--principal=oryx/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:36401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:58.434000 9692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:58.435137 9692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:58.444000 9701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:58.444062 9692 server_base.cc:1034] running on GCE node
W20260808 10:44:58.443977 9704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:58.444002 9702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:58.444797 9692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:58.446010 9692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:58.447198 9692 hybrid_clock.cc:648] HybridClock initialized: now 1786185898447179 us; error 32 us; skew 500 ppm
Aug 08 10:44:58 dist-test-slave-twwt krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185898, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:58.450117 9692 init.cc:377] Logged in from keytab as oryx/127.31.237.190@KRBTEST.COM (short username oryx)
I20260808 10:44:58.451095 9692 webserver.cc:460] Webserver started at http://127.31.237.190:45789/ using document root <none> and password file <none>
I20260808 10:44:58.451674 9692 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:58.451766 9692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:58.452009 9692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:58.454149 9692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "f908020858d84ab8a68a0c90fbc3ea5a"
format_stamp: "Formatted at 2026-08-08 10:44:58 on dist-test-slave-twwt"
server_key: "1d37690927d6e5c2e1157849a34e3ed5"
server_key_iv: "8fe4e5cec0b1ca7d05fb7f50fffd11ec"
server_key_version: "encryptionkey@0"
I20260808 10:44:58.454798 9692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "f908020858d84ab8a68a0c90fbc3ea5a"
format_stamp: "Formatted at 2026-08-08 10:44:58 on dist-test-slave-twwt"
server_key: "1d37690927d6e5c2e1157849a34e3ed5"
server_key_iv: "8fe4e5cec0b1ca7d05fb7f50fffd11ec"
server_key_version: "encryptionkey@0"
I20260808 10:44:58.459288 9692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:44:58.461896 9713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:58.463361 9692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:58.463524 9692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "f908020858d84ab8a68a0c90fbc3ea5a"
format_stamp: "Formatted at 2026-08-08 10:44:58 on dist-test-slave-twwt"
server_key: "1d37690927d6e5c2e1157849a34e3ed5"
server_key_iv: "8fe4e5cec0b1ca7d05fb7f50fffd11ec"
server_key_version: "encryptionkey@0"
I20260808 10:44:58.463660 9692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:58.483772 9692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:58.487534 9692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:58.487783 9692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:58.496179 9692 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:36401
I20260808 10:44:58.496204 9795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:36401 every 8 connection(s)
I20260808 10:44:58.497287 9692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:44:58.500285 9797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:58.503764 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 9692
I20260808 10:44:58.503924 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:44:58.504235 32694 external_mini_cluster.cc:1428] Setting key 371d43230dfccfe8cb3f5263896414ff
I20260808 10:44:58.506922 9797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a: Bootstrap starting.
I20260808 10:44:58.509776 9797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:58.510666 9797 log.cc:824] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:58.512794 9797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a: No bootstrap required, opened a new log
Aug 08 10:44:58 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
I20260808 10:44:58.516992 9797 raft_consensus.cc:348] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } }
I20260808 10:44:58.517290 9797 raft_consensus.cc:374] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:58.517356 9797 raft_consensus.cc:729] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f908020858d84ab8a68a0c90fbc3ea5a, State: Initialized, Role: FOLLOWER
I20260808 10:44:58.517863 9797 consensus_queue.cc:260] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } }
I20260808 10:44:58.517990 9797 raft_consensus.cc:388] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:44:58.518078 9797 raft_consensus.cc:482] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:44:58.518231 9797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:58.519316 9797 raft_consensus.cc:504] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } }
I20260808 10:44:58.519676 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:58.505676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36046 (local address 127.31.237.190:36401)
0808 10:44:58.506096 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:58.506116 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:58.506160 (+ 44us) server_negotiation.cc:407] Connection header received
0808 10:44:58.507005 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:58.507030 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:58.507477 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:58.507860 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:58.508569 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:58.510061 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:58.511010 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:58.511607 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:58.511763 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:58.514559 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:58.514565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:58.514578 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:58.514645 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:58.516446 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:58.516966 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:58.516972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:58.516976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:58.517116 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:58.517443 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:58.517446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:58.517450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:58.517940 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:58.518236 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:58.518524 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 10:44:58.518842 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":102,"threads_started":1}
I20260808 10:44:58.519672 9797 leader_election.cc:302] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f908020858d84ab8a68a0c90fbc3ea5a; no voters:
I20260808 10:44:58.520074 9797 leader_election.cc:288] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:44:58.520151 9805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:44:58.520357 9805 raft_consensus.cc:686] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [term 1 LEADER]: Becoming Leader. State: Replica: f908020858d84ab8a68a0c90fbc3ea5a, State: Running, Role: LEADER
I20260808 10:44:58.520736 9805 consensus_queue.cc:237] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } }
I20260808 10:44:58.521447 9797 sys_catalog.cc:564] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:58.522541 9809 sys_catalog.cc:455] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f908020858d84ab8a68a0c90fbc3ea5a. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } }
I20260808 10:44:58.522727 9809 sys_catalog.cc:458] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:58.523056 9807 sys_catalog.cc:455] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } }
I20260808 10:44:58.523149 9819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:44:58.523154 9807 sys_catalog.cc:458] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:58.526425 9819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:44:58.531989 9819 catalog_manager.cc:1347] Generated new cluster ID: 282a68a2c33345ef84eece0d29bd333f
I20260808 10:44:58.532056 9819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:44:58.541146 9819 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:44:58.542203 9819 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:44:58.557199 9819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a: Generated new TSK 0
I20260808 10:44:58.557960 9819 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.129.keytab.
Entry for principal oryx/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.129.keytab.
I20260808 10:44:58.592965 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:36401
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.129.keytab
--principal=oryx/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:58.713817 9846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:58.714098 9846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:58.714143 9846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:58.718287 9846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:58.718362 9846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:58.718456 9846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:44:58.724243 9846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.129.keytab
--principal=oryx/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:58.725421 9846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:58.726549 9846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:58.737108 9855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:58.737131 9859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:58.737268 9846 server_base.cc:1034] running on GCE node
W20260808 10:44:58.737131 9856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:58.737821 9846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:58.738520 9846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:58.739707 9846 hybrid_clock.cc:648] HybridClock initialized: now 1786185898739671 us; error 48 us; skew 500 ppm
Aug 08 10:44:58 dist-test-slave-twwt krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185898, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:58.742722 9846 init.cc:377] Logged in from keytab as oryx/127.31.237.129@KRBTEST.COM (short username oryx)
I20260808 10:44:58.743660 9846 webserver.cc:460] Webserver started at http://127.31.237.129:46675/ using document root <none> and password file <none>
I20260808 10:44:58.744251 9846 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:58.744324 9846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:58.744525 9846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:58.746827 9846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "8cff3352097c4b25a3805d484ad98c80"
format_stamp: "Formatted at 2026-08-08 10:44:58 on dist-test-slave-twwt"
server_key: "f1f7a4186aa0604f9d8036f11f542d6d"
server_key_iv: "8df7198ecdf0b93980050e5b4ae60cdc"
server_key_version: "encryptionkey@0"
I20260808 10:44:58.747401 9846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "8cff3352097c4b25a3805d484ad98c80"
format_stamp: "Formatted at 2026-08-08 10:44:58 on dist-test-slave-twwt"
server_key: "f1f7a4186aa0604f9d8036f11f542d6d"
server_key_iv: "8df7198ecdf0b93980050e5b4ae60cdc"
server_key_version: "encryptionkey@0"
I20260808 10:44:58.751554 9846 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:44:58.754276 9871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:58.755667 9846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:58.755833 9846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "8cff3352097c4b25a3805d484ad98c80"
format_stamp: "Formatted at 2026-08-08 10:44:58 on dist-test-slave-twwt"
server_key: "f1f7a4186aa0604f9d8036f11f542d6d"
server_key_iv: "8df7198ecdf0b93980050e5b4ae60cdc"
server_key_version: "encryptionkey@0"
I20260808 10:44:58.756006 9846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:58.789695 9846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:58.793192 9846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:58.793432 9846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:58.794099 9846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:58.795047 9846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:58.795123 9846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:58.795198 9846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:58.795243 9846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:58.805063 9846 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:32989
I20260808 10:44:58.805140 10064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:32989 every 8 connection(s)
I20260808 10:44:58.806085 9846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:44:58.811632 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 9846
I20260808 10:44:58.811784 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:44:58.812095 32694 external_mini_cluster.cc:1428] Setting key dbdd8e32408a4a65b7aa1cdb357e0747
Aug 08 10:44:58 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185898, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
I20260808 10:44:58.821660 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:58.808186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:55985 (local address 127.31.237.190:36401)
0808 10:44:58.808342 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:58.808348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:58.809244 (+ 896us) server_negotiation.cc:407] Connection header received
0808 10:44:58.810749 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:58.810753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:58.810812 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:58.810933 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:58.812324 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:58.813032 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:58.814337 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:58.814576 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:58.814734 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:58.817706 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:58.817709 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:58.817711 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:58.817736 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:58.819328 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:58.819921 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:58.819925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:58.819927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:58.820006 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:58.820374 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:58.820377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:58.820379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:58.820590 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:58.820704 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:58.821346 (+ 642us) server_negotiation.cc:299] Negotiation successful
0808 10:44:58.821487 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:44:58.822731 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:58.808593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.129:55985)
0808 10:44:58.809083 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:58.809128 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:58.810481 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:58.811126 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:58.811133 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:58.811524 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:58.812163 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:58.812186 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:58.813169 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:58.813173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:58.814203 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:58.814214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:58.814781 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:58.814785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:58.814962 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:58.815626 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:58.815644 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:58.817515 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:58.819472 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:58.819479 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:58.819498 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:58.819784 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:58.820105 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:58.820109 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:58.820111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:58.820267 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:58.820704 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:58.820713 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:58.821078 (+ 365us) client_negotiation.cc:769] Sending connection context
0808 10:44:58.821290 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 10:44:58.821666 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":125,"threads_started":1}
WARNING: no policy specified for oryx/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:44:58.824223 10066 heartbeater.cc:347] Connected to a master server at 127.31.237.190:36401
I20260808 10:44:58.824514 10066 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:58.825091 10066 heartbeater.cc:511] Master 127.31.237.190:36401 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.237.130@KRBTEST.COM" created.
I20260808 10:44:58.826881 9737 ts_manager.cc:194] Registered new tserver with Master: 8cff3352097c4b25a3805d484ad98c80 (127.31.237.129:32989)
I20260808 10:44:58.828403 9737 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:55985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.130.keytab.
Entry for principal oryx/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.130.keytab.
I20260808 10:44:58.841981 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:36401
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.130.keytab
--principal=oryx/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:58.958465 10073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:58.958755 10073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:58.958801 10073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:58.962939 10073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:58.963011 10073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:58.963073 10073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:44:58.968477 10073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.130.keytab
--principal=oryx/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:58.969650 10073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:58.970688 10073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:58.980602 10083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:58.980590 10081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:58.980590 10087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:58.980715 10073 server_base.cc:1034] running on GCE node
I20260808 10:44:58.981292 10073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:58.981938 10073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:58.983129 10073 hybrid_clock.cc:648] HybridClock initialized: now 1786185898983091 us; error 46 us; skew 500 ppm
Aug 08 10:44:58 dist-test-slave-twwt krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185898, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:58.986183 10073 init.cc:377] Logged in from keytab as oryx/127.31.237.130@KRBTEST.COM (short username oryx)
I20260808 10:44:58.987250 10073 webserver.cc:460] Webserver started at http://127.31.237.130:37611/ using document root <none> and password file <none>
I20260808 10:44:58.987908 10073 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:58.987980 10073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:58.988234 10073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:58.990435 10073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "7910b12be2c048c1bde2a069315e9c32"
format_stamp: "Formatted at 2026-08-08 10:44:58 on dist-test-slave-twwt"
server_key: "f603b93946819bbe72f90287259fcf87"
server_key_iv: "981a213b8752cb1f7ec0a5dd884ff97f"
server_key_version: "encryptionkey@0"
I20260808 10:44:58.991073 10073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "7910b12be2c048c1bde2a069315e9c32"
format_stamp: "Formatted at 2026-08-08 10:44:58 on dist-test-slave-twwt"
server_key: "f603b93946819bbe72f90287259fcf87"
server_key_iv: "981a213b8752cb1f7ec0a5dd884ff97f"
server_key_version: "encryptionkey@0"
I20260808 10:44:58.995064 10073 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:44:58.997689 10097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:58.999145 10073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:58.999320 10073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "7910b12be2c048c1bde2a069315e9c32"
format_stamp: "Formatted at 2026-08-08 10:44:58 on dist-test-slave-twwt"
server_key: "f603b93946819bbe72f90287259fcf87"
server_key_iv: "981a213b8752cb1f7ec0a5dd884ff97f"
server_key_version: "encryptionkey@0"
I20260808 10:44:58.999452 10073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:59.019337 10073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:59.023535 10073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:59.023792 10073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:59.024495 10073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:59.025471 10073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:59.025556 10073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:59.025638 10073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:59.025689 10073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:59.035771 10073 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:35331
I20260808 10:44:59.035825 10272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:35331 every 8 connection(s)
I20260808 10:44:59.036892 10073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:44:59.040027 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 10073
I20260808 10:44:59.040158 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:44:59.040545 32694 external_mini_cluster.cc:1428] Setting key dc2993136cabb19458d328ad0fb5e5ad
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185898, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:44:59.054858 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.039076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:53441 (local address 127.31.237.190:36401)
0808 10:44:59.039246 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.039251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.040343 (+ 1092us) server_negotiation.cc:407] Connection header received
0808 10:44:59.042099 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.042104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.042163 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.042233 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.044000 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.044586 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.046366 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.046815 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.046962 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.049950 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.049953 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.049955 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.049982 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.051992 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.052736 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.052741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.052744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.052854 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.053365 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.053370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.053373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.053687 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.053815 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.054542 (+ 727us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.054724 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.237.131@KRBTEST.COM" created.
I20260808 10:44:59.055979 10278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.039443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.130:53441)
0808 10:44:59.040135 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.040203 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.041859 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.042460 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.042469 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.043071 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.043823 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.043849 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.044731 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.044737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.046207 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.046218 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.046987 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.046990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.047146 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.047832 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.047855 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.049742 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.052131 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.052138 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.052175 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.052565 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.052993 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.052996 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.052998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.053203 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.053821 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.053851 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.054232 (+ 381us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.054498 (+ 266us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.054872 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":141,"threads_started":1}
I20260808 10:44:59.057389 10275 heartbeater.cc:347] Connected to a master server at 127.31.237.190:36401
I20260808 10:44:59.057657 10275 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:59.058329 10275 heartbeater.cc:511] Master 127.31.237.190:36401 requested a full tablet report, sending...
I20260808 10:44:59.059674 9737 ts_manager.cc:194] Registered new tserver with Master: 7910b12be2c048c1bde2a069315e9c32 (127.31.237.130:35331)
I20260808 10:44:59.060606 9737 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:53441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.131.keytab.
Entry for principal oryx/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.131.keytab.
I20260808 10:44:59.073495 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:36401
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.131.keytab
--principal=oryx/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:59.190712 10282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:59.190994 10282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:59.191067 10282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:59.195178 10282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:59.195243 10282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:59.195338 10282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:44:59.200764 10282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.131.keytab
--principal=oryx/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:36401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:59.201920 10282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:59.203123 10282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:59.213096 10288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:59.213096 10295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:59.213107 10290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:59.213376 10282 server_base.cc:1034] running on GCE node
I20260808 10:44:59.213843 10282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:59.214468 10282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:59.215663 10282 hybrid_clock.cc:648] HybridClock initialized: now 1786185899215629 us; error 50 us; skew 500 ppm
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185899, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:59.218546 10282 init.cc:377] Logged in from keytab as oryx/127.31.237.131@KRBTEST.COM (short username oryx)
I20260808 10:44:59.219542 10282 webserver.cc:460] Webserver started at http://127.31.237.131:35701/ using document root <none> and password file <none>
I20260808 10:44:59.220151 10282 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:59.220243 10282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:59.220495 10282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:59.222743 10282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "11f5c1015791423bbf99231d8cc6f903"
format_stamp: "Formatted at 2026-08-08 10:44:59 on dist-test-slave-twwt"
server_key: "82877612b4e0534263c1301fa3b2fb2e"
server_key_iv: "0b8e464b0567dc50e6e08a5140dc5f61"
server_key_version: "encryptionkey@0"
I20260808 10:44:59.223325 10282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "11f5c1015791423bbf99231d8cc6f903"
format_stamp: "Formatted at 2026-08-08 10:44:59 on dist-test-slave-twwt"
server_key: "82877612b4e0534263c1301fa3b2fb2e"
server_key_iv: "0b8e464b0567dc50e6e08a5140dc5f61"
server_key_version: "encryptionkey@0"
I20260808 10:44:59.227098 10282 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 10:44:59.229547 10303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:59.230981 10282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:44:59.231123 10282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "11f5c1015791423bbf99231d8cc6f903"
format_stamp: "Formatted at 2026-08-08 10:44:59 on dist-test-slave-twwt"
server_key: "82877612b4e0534263c1301fa3b2fb2e"
server_key_iv: "0b8e464b0567dc50e6e08a5140dc5f61"
server_key_version: "encryptionkey@0"
I20260808 10:44:59.231240 10282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:59.248458 10282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:59.252122 10282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:59.252353 10282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:59.253015 10282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:44:59.253907 10282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:44:59.253980 10282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:59.254058 10282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:44:59.254103 10282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:59.264001 10282 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:45425
I20260808 10:44:59.264027 10479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:45425 every 8 connection(s)
I20260808 10:44:59.264925 10282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:44:59.271407 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 10282
I20260808 10:44:59.271554 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:44:59.271842 32694 external_mini_cluster.cc:1428] Setting key a8ad5c389eca796849eb1a358998d104
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185899, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
I20260808 10:44:59.279155 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.266860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:35997 (local address 127.31.237.190:36401)
0808 10:44:59.267017 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.267020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.267922 (+ 902us) server_negotiation.cc:407] Connection header received
0808 10:44:59.269366 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.269369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.269416 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.269493 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.270764 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.271330 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.272833 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.273111 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.273237 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.275798 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.275801 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.275803 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.275827 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.277307 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.277780 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.277783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.277784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.277857 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.278153 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.278155 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.278156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.278320 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.278412 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.278938 (+ 526us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.279033 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:44:59.280010 10484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.267244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.131:35997)
0808 10:44:59.267775 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.267819 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.269164 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.269661 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.269668 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.270022 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.270563 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.270578 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.271480 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.271484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.272677 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.272689 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.273265 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.273268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.273512 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.274182 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.274200 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.275648 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.277428 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.277433 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.277445 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.277686 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.277933 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.277935 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.277937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.278068 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.278435 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.278441 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.278727 (+ 286us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.278893 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.279150 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":120,"threads_started":1}
I20260808 10:44:59.281149 10480 heartbeater.cc:347] Connected to a master server at 127.31.237.190:36401
I20260808 10:44:59.281417 10480 heartbeater.cc:464] Registering TS with master...
I20260808 10:44:59.281927 10480 heartbeater.cc:511] Master 127.31.237.190:36401 requested a full tablet report, sending...
I20260808 10:44:59.282945 9737 ts_manager.cc:194] Registered new tserver with Master: 11f5c1015791423bbf99231d8cc6f903 (127.31.237.131:45425)
I20260808 10:44:59.283535 9737 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:35997
I20260808 10:44:59.285981 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.237.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.237.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.237.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.189.keytab.
Entry for principal oryx/127.31.237.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.189.keytab.
I20260808 10:44:59.315760 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.189:35269
--webserver_interface=127.31.237.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.189.keytab
--principal=oryx/127.31.237.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:36401,127.31.237.189:35269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:44:59.436136 10489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:59.436407 10489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:59.436472 10489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:59.440467 10489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:59.440521 10489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:59.440541 10489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:59.440557 10489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:59.440599 10489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:59.446010 10489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:36401,127.31.237.189:35269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.189.keytab
--principal=oryx/127.31.237.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.189:35269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.237.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
W20260808 10:44:59.446233 10489 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.31.237.190:36401,127.31.237.189:35269'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 10:44:59.447292 10489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:59.448359 10489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:59.457157 10502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:59.457166 10497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:59.457253 10499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:59.457401 10489 server_base.cc:1034] running on GCE node
I20260808 10:44:59.458175 10489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:59.459419 10489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:59.460588 10489 hybrid_clock.cc:648] HybridClock initialized: now 1786185899460558 us; error 64 us; skew 500 ppm
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185899, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:59.463519 10489 init.cc:377] Logged in from keytab as oryx/127.31.237.189@KRBTEST.COM (short username oryx)
I20260808 10:44:59.464443 10489 webserver.cc:460] Webserver started at http://127.31.237.189:44603/ using document root <none> and password file <none>
I20260808 10:44:59.465021 10489 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:59.465114 10489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:59.465354 10489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:59.467495 10489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/data/instance:
uuid: "e2ee44ced7704478a10c8c32f9eb9570"
format_stamp: "Formatted at 2026-08-08 10:44:59 on dist-test-slave-twwt"
server_key: "e58729da6b0284bbd09e511a4bb03d04"
server_key_iv: "040a34c30fcd70ecfc11aa3ef0c58fb9"
server_key_version: "encryptionkey@0"
I20260808 10:44:59.468075 10489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/wal/instance:
uuid: "e2ee44ced7704478a10c8c32f9eb9570"
format_stamp: "Formatted at 2026-08-08 10:44:59 on dist-test-slave-twwt"
server_key: "e58729da6b0284bbd09e511a4bb03d04"
server_key_iv: "040a34c30fcd70ecfc11aa3ef0c58fb9"
server_key_version: "encryptionkey@0"
I20260808 10:44:59.471830 10489 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 10:44:59.474285 10514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:59.475636 10489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:44:59.475811 10489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/wal
uuid: "e2ee44ced7704478a10c8c32f9eb9570"
format_stamp: "Formatted at 2026-08-08 10:44:59 on dist-test-slave-twwt"
server_key: "e58729da6b0284bbd09e511a4bb03d04"
server_key_iv: "040a34c30fcd70ecfc11aa3ef0c58fb9"
server_key_version: "encryptionkey@0"
I20260808 10:44:59.475930 10489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:59.495945 10489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:59.499352 10489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:59.499550 10489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:59.507259 10489 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.189:35269
I20260808 10:44:59.507309 10598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.189:35269 every 8 connection(s)
I20260808 10:44:59.508253 10489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/data/info.pb
I20260808 10:44:59.511147 10602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:59.513214 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 10489
I20260808 10:44:59.515431 10602 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.237.189@KRBTEST.COM
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185899, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.189@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
I20260808 10:44:59.528604 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.517942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36062 (local address 127.31.237.190:36401)
0808 10:44:59.518096 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.518100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.518464 (+ 364us) server_negotiation.cc:407] Connection header received
0808 10:44:59.518583 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.518587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.518660 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.518748 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.519442 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.519965 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.521036 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.521273 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.521387 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.524465 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.524470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.524474 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.524505 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.526243 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.527013 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.527017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.527018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.527111 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.527433 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.527437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.527439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.527720 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.527843 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.528252 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.528401 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:44:59.529448 10608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.517928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.0.0.1:36062)
0808 10:44:59.518324 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.518375 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.518493 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.518803 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.518808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.519135 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.519309 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.519319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.520105 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.520112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.520886 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.520896 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.521427 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.521431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.521537 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.522052 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.522068 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.524309 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.526369 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.526376 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.526384 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.526903 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.527232 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.527234 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.527236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.527327 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.527850 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.527856 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.528001 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.528185 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.528638 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":156,"thread_start_us":148,"threads_started":1}
I20260808 10:44:59.529454 10607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.514912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.31.237.189:35269)
0808 10:44:59.515330 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.515350 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.515392 (+ 42us) server_negotiation.cc:407] Connection header received
0808 10:44:59.516214 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.516241 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.516661 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.517029 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.517851 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.519162 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.520120 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.520622 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.520735 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.523738 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.523748 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.523762 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.523790 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.526013 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.526665 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.526672 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.526673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.526812 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.527174 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.527178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.527180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.527720 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.527938 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.528187 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.528559 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":273,"thread_start_us":112,"threads_started":1}
I20260808 10:44:59.531755 10602 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185899, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.189@KRBTEST.COM for oryx/127.31.237.189@KRBTEST.COM
I20260808 10:44:59.540246 10608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.532277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:35269 (local address 127.0.0.1:59732)
0808 10:44:59.532406 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.532423 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.532504 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.532728 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.532731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.532911 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.532995 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.533000 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.534097 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.534102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.534736 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.534745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.535196 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.535199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.535360 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.535861 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.535875 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.537125 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.538944 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.538947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.538948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.539129 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.539390 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.539392 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.539394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.539465 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.539862 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.539865 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.539941 (+ 76us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.540026 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.540124 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":34}
I20260808 10:44:59.540341 10607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.532362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59732 (local address 127.31.237.189:35269)
0808 10:44:59.532476 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.532480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.532493 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:44:59.532586 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.532589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.532622 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.532706 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.533113 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.533984 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.534859 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.535089 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.535198 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.537251 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.537255 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.537257 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.537288 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.538841 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.539219 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.539222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.539223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.539306 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.539545 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.539548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.539549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.539732 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.539856 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.540095 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.540193 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":46}
I20260808 10:44:59.543558 10602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: Bootstrap starting.
I20260808 10:44:59.545857 10602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:59.546576 10602 log.cc:824] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:59.548552 10602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: No bootstrap required, opened a new log
I20260808 10:44:59.551667 10602 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } }
I20260808 10:44:59.551870 10602 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:59.551904 10602 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2ee44ced7704478a10c8c32f9eb9570, State: Initialized, Role: FOLLOWER
I20260808 10:44:59.552287 10602 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } }
I20260808 10:44:59.552701 10610 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } } }
I20260808 10:44:59.552829 10610 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:59.553433 10602 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 10:44:59.558670 10624 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:59.558765 10624 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:59.561440 10489 master_runner.cc:428] Detected that this master e2ee44ced7704478a10c8c32f9eb9570 is joining an existing cluster
I20260808 10:44:59.561503 10489 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.237.189:35269
I20260808 10:44:59.573243 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.564212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36074 (local address 127.31.237.190:36401)
0808 10:44:59.564332 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.564337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.564541 (+ 204us) server_negotiation.cc:407] Connection header received
0808 10:44:59.564949 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.564952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.565001 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.565082 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.565644 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.566176 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.567639 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.567863 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.567982 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.569633 (+ 1651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.569637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.569640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.569671 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.571464 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.571969 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.571973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.571975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.572064 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.572372 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.572375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.572377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.572571 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.572681 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.572968 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.573083 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 10:44:59.573417 10607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.564102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59740 (local address 127.31.237.189:35269)
0808 10:44:59.564332 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.564337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.564545 (+ 208us) server_negotiation.cc:407] Connection header received
0808 10:44:59.564661 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.564664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.564714 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.564810 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.565437 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.566268 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.567602 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.567945 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.568051 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.569972 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.569977 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.569981 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.570008 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.571780 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.572204 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.572208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.572209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.572285 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.572557 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.572560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.572562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.572752 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.572818 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.573114 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.573221 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":166}
I20260808 10:44:59.573573 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.563957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:35269 (local address 127.0.0.1:59740)
0808 10:44:59.564472 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.564486 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.564584 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.564859 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.564862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.565114 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.565303 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.565311 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.566366 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.566369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.567484 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.567495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.568174 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.568178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.568264 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.568919 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.568935 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.569477 (+ 542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.571887 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.571890 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.571891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.572103 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.572389 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.572392 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.572394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.572471 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.572888 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.572890 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.572964 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.573282 (+ 318us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.573381 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":152,"thread_start_us":113,"threads_started":1}
I20260808 10:44:59.573261 10630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.563957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.0.0.1:36074)
0808 10:44:59.564436 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.564452 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.564571 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.565128 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.565131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.565346 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.565525 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.565532 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.566299 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.566306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.567509 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.567517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.568492 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.568496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.568617 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.569080 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.569095 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.569485 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.571592 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.571595 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.571597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.571874 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.572161 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.572166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.572171 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.572265 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.572672 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.572675 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.572799 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.572911 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.573053 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":104,"thread_start_us":110,"threads_started":1}
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185898, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.190@KRBTEST.COM for oryx/127.31.237.189@KRBTEST.COM
I20260808 10:44:59.590137 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.578923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:35269 (local address 127.0.0.1:59746)
0808 10:44:59.579403 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.579457 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.579586 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.579876 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.579880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.580400 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.580575 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.580582 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.581586 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.581590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.582396 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.582408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.583097 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.583101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.583222 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.583833 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.583848 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.585652 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.587854 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.587864 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.587871 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.588184 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.588511 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.588514 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.588516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.588613 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.589404 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.589412 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.589562 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.589725 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.589942 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":179,"threads_started":1}
I20260808 10:44:59.590853 10607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.578913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59746 (local address 127.31.237.189:35269)
0808 10:44:59.579038 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.579043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.579517 (+ 474us) server_negotiation.cc:407] Connection header received
0808 10:44:59.579721 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.579725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.579776 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.579856 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.580686 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.581452 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.582517 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.582870 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.582975 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.585838 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.585843 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.585849 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.585880 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.587731 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.588302 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.588308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.588313 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.588410 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.588715 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.588721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.588725 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.589216 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.589363 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.590545 (+ 1182us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.590696 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:44:59.592446 9737 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.237.189:35269: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570"
member_type: NON_VOTER
last_known_addr {
host: "127.31.237.189"
port: 35269
}
attrs {
promote: true
}
}
dest_uuid: "f908020858d84ab8a68a0c90fbc3ea5a"
cas_config_opid_index: -1
I20260808 10:44:59.593326 9737 consensus_queue.cc:237] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: NON_VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.31.237.188@KRBTEST.COM; defaulting to no policy
I20260808 10:44:59.595736 10636 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e2ee44ced7704478a10c8c32f9eb9570 (127.31.237.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: NON_VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: true } } }
I20260808 10:44:59.596652 10560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.237.188@KRBTEST.COM" created.
I20260808 10:44:59.598974 10636 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.237.189:35269
I20260808 10:44:59.599426 10635 sys_catalog.cc:455] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: NON_VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: true } } }
I20260808 10:44:59.599612 10635 sys_catalog.cc:458] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:59.599777 10560 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 FOLLOWER]: Refusing update from remote peer f908020858d84ab8a68a0c90fbc3ea5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 10:44:59.600165 10489 master.cc:537] Master@127.31.237.189:35269 shutting down...
I20260808 10:44:59.600857 10635 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: NON_VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:44:59.603295 9719 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 10:44:59.604358 9719 consensus_peers.cc:487] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a -> Peer e2ee44ced7704478a10c8c32f9eb9570 (127.31.237.189:35269): Couldn't send request to peer e2ee44ced7704478a10c8c32f9eb9570. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 10:44:59.606078 10489 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:59.606416 10489 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:59.606658 10489 tablet_replica.cc:330] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: stopping tablet replica
I20260808 10:44:59.612298 10489 master.cc:552] Master@127.31.237.189:35269 shutdown complete.
I20260808 10:44:59.612373 10489 master_runner.cc:305] Clearing existing system tablet
I20260808 10:44:59.612601 10489 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:59.612676 10489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:59.614727 10489 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:44:59.615692 10642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:59.616001 10489 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:59.616129 10489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:44:59.616182 10489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/wal
uuid: "e2ee44ced7704478a10c8c32f9eb9570"
format_stamp: "Formatted at 2026-08-08 10:44:59 on dist-test-slave-twwt"
server_key: "e58729da6b0284bbd09e511a4bb03d04"
server_key_iv: "040a34c30fcd70ecfc11aa3ef0c58fb9"
server_key_version: "encryptionkey@0"
I20260808 10:44:59.618345 10489 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.237.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.188.keytab.
Entry for principal oryx/127.31.237.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.188.keytab.
I20260808 10:44:59.621591 10489 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 10:44:59.621665 10489 log.cc:1196] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 10:44:59.621915 10489 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: Deleting consensus metadata
I20260808 10:44:59.622119 10489 master_runner.cc:315] Copying system tablet from 127.31.237.190:36401
I20260808 10:44:59.622191 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.188:42141
--webserver_interface=127.31.237.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.188.keytab
--principal=oryx/127.31.237.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:36401,127.31.237.189:35269,127.31.237.188:42141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:44:59.623466 10489 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: tablet copy: Beginning tablet copy session from remote peer at address 127.31.237.190:36401
I20260808 10:44:59.636286 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.624114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36076 (local address 127.31.237.190:36401)
0808 10:44:59.624252 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.624256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.626974 (+ 2718us) server_negotiation.cc:407] Connection header received
0808 10:44:59.627126 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.627130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.627192 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.627291 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.628024 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.628860 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.630247 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.630599 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.630746 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.632304 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.632308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.632311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.632339 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.634161 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.634741 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.634745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.634747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.634845 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.635193 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.635197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.635198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.635444 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.635547 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.635923 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.636107 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:44:59.636427 10658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.624069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.0.0.1:36076)
0808 10:44:59.626841 (+ 2772us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.626860 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.627014 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.627344 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.627350 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.627636 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.627817 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.627829 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.629001 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.629009 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.630096 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.630112 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.630792 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.630798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.630935 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.631602 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.631625 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.632135 (+ 510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.634293 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.634299 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.634304 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.634639 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.634957 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.634962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.634966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.635081 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.635567 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.635573 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.635708 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.635845 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.636106 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2664,"thread_start_us":92,"threads_started":1}
I20260808 10:44:59.637251 9767 tablet_copy_service.cc:140] P f908020858d84ab8a68a0c90fbc3ea5a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e2ee44ced7704478a10c8c32f9eb9570 ({username='oryx', principal='oryx/127.31.237.189@KRBTEST.COM'} at 127.0.0.1:36076)
I20260808 10:44:59.637459 9767 tablet_copy_service.cc:161] P f908020858d84ab8a68a0c90fbc3ea5a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e2ee44ced7704478a10c8c32f9eb9570 at {username='oryx', principal='oryx/127.31.237.189@KRBTEST.COM'} at 127.0.0.1:36076: session id = e2ee44ced7704478a10c8c32f9eb9570-00000000000000000000000000000000
I20260808 10:44:59.639406 9767 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 10:44:59.641316 10489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 10:44:59.643911 10489 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: tablet copy: Starting download of 0 data blocks...
I20260808 10:44:59.644119 10489 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: tablet copy: Starting download of 1 WAL segments...
I20260808 10:44:59.645977 10489 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 10:44:59.647925 9767 tablet_copy_service.cc:342] P f908020858d84ab8a68a0c90fbc3ea5a: Request end of tablet copy session e2ee44ced7704478a10c8c32f9eb9570-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.237.189@KRBTEST.COM'} at 127.0.0.1:36076
I20260808 10:44:59.648058 9767 tablet_copy_service.cc:433] P f908020858d84ab8a68a0c90fbc3ea5a: ending tablet copy session e2ee44ced7704478a10c8c32f9eb9570-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e2ee44ced7704478a10c8c32f9eb9570
I20260808 10:44:59.656010 10489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:59.657645 10659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:59.659387 10660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:59.659911 10662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:59.660117 10489 server_base.cc:1034] running on GCE node
I20260808 10:44:59.660440 10489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:59.660745 10489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:59.661876 10489 hybrid_clock.cc:648] HybridClock initialized: now 1786185899661865 us; error 35 us; skew 500 ppm
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185899, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:59.663623 10489 init.cc:377] Logged in from keytab as oryx/127.31.237.189@KRBTEST.COM (short username oryx)
I20260808 10:44:59.664077 10489 webserver.cc:460] Webserver started at http://127.31.237.189:43467/ using document root <none> and password file <none>
I20260808 10:44:59.664260 10489 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:59.664292 10489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:59.665614 10489 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:44:59.666718 10675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:59.666954 10489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:44:59.667035 10489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/wal
uuid: "e2ee44ced7704478a10c8c32f9eb9570"
format_stamp: "Formatted at 2026-08-08 10:44:59 on dist-test-slave-twwt"
server_key: "e58729da6b0284bbd09e511a4bb03d04"
server_key_iv: "040a34c30fcd70ecfc11aa3ef0c58fb9"
server_key_version: "encryptionkey@0"
I20260808 10:44:59.667120 10489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:59.685902 10489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:59.688268 10489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:59.693503 10489 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.189:35269
I20260808 10:44:59.693540 10760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.189:35269 every 8 connection(s)
I20260808 10:44:59.694108 10489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-1/data/info.pb
I20260808 10:44:59.696099 10762 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:44:59.697324 10762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: Bootstrap starting.
I20260808 10:44:59.709452 10762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:44:59.709972 10762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: Bootstrap complete.
I20260808 10:44:59.710780 10762 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: NON_VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: true } }
I20260808 10:44:59.711018 10762 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e2ee44ced7704478a10c8c32f9eb9570, State: Initialized, Role: LEARNER
I20260808 10:44:59.711182 10762 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: NON_VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: true } }
I20260808 10:44:59.711709 10768 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: NON_VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: true } } }
I20260808 10:44:59.711833 10768 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: This master's current role is: LEARNER
I20260808 10:44:59.711916 10762 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:44:59.715667 10797 catalog_manager.cc:1259] Loaded cluster ID: 282a68a2c33345ef84eece0d29bd333f
I20260808 10:44:59.715736 10797 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: loading cluster ID for follower catalog manager: success
I20260808 10:44:59.717895 10797 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: acquiring CA information for follower catalog manager: success
I20260808 10:44:59.719347 10797 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 10:44:59.761649 10657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:44:59.761940 10657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:44:59.761983 10657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:44:59.766026 10657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:44:59.766121 10657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:44:59.766142 10657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:44:59.766160 10657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:44:59.766175 10657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:44:59.771822 10657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:36401,127.31.237.189:35269,127.31.237.188:42141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.188.keytab
--principal=oryx/127.31.237.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.188:42141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.237.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:44:59.772992 10657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:44:59.774027 10657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:44:59.783330 10807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:59.783324 10812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:44:59.783324 10808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:44:59.783488 10657 server_base.cc:1034] running on GCE node
I20260808 10:44:59.784240 10657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:44:59.785439 10657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:44:59.786670 10657 hybrid_clock.cc:648] HybridClock initialized: now 1786185899786623 us; error 54 us; skew 500 ppm
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185899, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:44:59.789561 10657 init.cc:377] Logged in from keytab as oryx/127.31.237.188@KRBTEST.COM (short username oryx)
I20260808 10:44:59.790530 10657 webserver.cc:460] Webserver started at http://127.31.237.188:40823/ using document root <none> and password file <none>
I20260808 10:44:59.791157 10657 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:59.791257 10657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:59.791514 10657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:44:59.793685 10657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/data/instance:
uuid: "56098355de674fc0ae7dbdf27c5dc2e3"
format_stamp: "Formatted at 2026-08-08 10:44:59 on dist-test-slave-twwt"
server_key: "0cdea6d9d5e6c439b59f14f91c126035"
server_key_iv: "d5b4e8e86ba273a3ff8ce4419c492c5b"
server_key_version: "encryptionkey@0"
I20260808 10:44:59.794301 10657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/wal/instance:
uuid: "56098355de674fc0ae7dbdf27c5dc2e3"
format_stamp: "Formatted at 2026-08-08 10:44:59 on dist-test-slave-twwt"
server_key: "0cdea6d9d5e6c439b59f14f91c126035"
server_key_iv: "d5b4e8e86ba273a3ff8ce4419c492c5b"
server_key_version: "encryptionkey@0"
I20260808 10:44:59.798278 10657 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 10:44:59.800808 10822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:59.802186 10657 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:44:59.802335 10657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/wal
uuid: "56098355de674fc0ae7dbdf27c5dc2e3"
format_stamp: "Formatted at 2026-08-08 10:44:59 on dist-test-slave-twwt"
server_key: "0cdea6d9d5e6c439b59f14f91c126035"
server_key_iv: "d5b4e8e86ba273a3ff8ce4419c492c5b"
server_key_version: "encryptionkey@0"
I20260808 10:44:59.802445 10657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:59.819775 10657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:44:59.823370 10657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:44:59.823585 10657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:44:59.831676 10066 heartbeater.cc:503] Master 127.31.237.190:36401 was elected leader, sending a full tablet report...
I20260808 10:44:59.832310 10657 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.188:42141
I20260808 10:44:59.832623 10913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.188:42141 every 8 connection(s)
I20260808 10:44:59.833598 10657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/data/info.pb
I20260808 10:44:59.836849 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:44:59.840698 10915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:44:59.842110 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 10657
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185899, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.188@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.237.188@KRBTEST.COM
I20260808 10:44:59.854527 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.842689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36082 (local address 127.31.237.190:36401)
0808 10:44:59.842893 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.842899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.843743 (+ 844us) server_negotiation.cc:407] Connection header received
0808 10:44:59.844605 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.844608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.844668 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.844774 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.845812 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.846319 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.847444 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.847694 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.847771 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.850581 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.850593 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.850595 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.850661 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.852170 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.852883 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.852887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.852889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.852981 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.853330 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.853334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.853336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.853524 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.853643 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.854222 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.854348 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:44:59.855492 10918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.843086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47792 (local address 127.31.237.188:42141)
0808 10:44:59.843596 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.843614 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.843653 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:44:59.844392 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.844411 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.844606 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.844787 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.845538 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.846616 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.847494 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.847975 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.848074 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.851166 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.851180 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.851190 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.851211 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.852984 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.853433 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.853438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.853439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.853512 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.853797 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.853800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.853802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.854046 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.854269 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.854464 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.854673 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":394,"spinlock_wait_cycles":54016,"thread_start_us":192,"threads_started":1}
I20260808 10:44:59.855489 10917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.843081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.0.0.1:36082)
0808 10:44:59.843614 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.843652 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.844401 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.844852 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.844857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.845302 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.845662 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.845679 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.846456 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.846460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.847314 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.847323 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.847833 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.847838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.848032 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.848835 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.848851 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.850430 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.852383 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.852394 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.852405 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.852684 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.853080 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.853083 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.853085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.853228 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.853705 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.853711 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.853982 (+ 271us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.854178 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.854501 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":1,"thread_start_us":153,"threads_started":1}
I20260808 10:44:59.856815 10915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185899, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.188@KRBTEST.COM for oryx/127.31.237.189@KRBTEST.COM
I20260808 10:44:59.865142 10917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.857314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:35269 (local address 127.0.0.1:59760)
0808 10:44:59.857711 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.857722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.857788 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.857991 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.857993 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.858115 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.858252 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.858257 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.859220 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.859222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.859802 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.859815 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.860239 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.860241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.860313 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.860783 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.860795 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.862026 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.863662 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.863666 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.863668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.863871 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.864160 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.864163 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.864165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.864248 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.864644 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.864647 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.864744 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.864859 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.864987 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330}
I20260808 10:44:59.865195 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.857435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59760 (local address 127.31.237.189:35269)
0808 10:44:59.857686 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.857689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.857757 (+ 68us) server_negotiation.cc:407] Connection header received
0808 10:44:59.857849 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.857851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.857890 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.857964 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.858356 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.859123 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.859924 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.860145 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.860228 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.862174 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.862177 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.862179 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.862201 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.863550 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.863974 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.863981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.863983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.864065 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.864362 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.864367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.864368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.864544 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.864653 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.864914 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.865029 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":110,"threads_started":1}
I20260808 10:44:59.865937 10915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185899, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.188@KRBTEST.COM for oryx/127.31.237.188@KRBTEST.COM
I20260808 10:44:59.873548 10917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.866497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42141 (local address 127.0.0.1:47802)
0808 10:44:59.866672 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.866686 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.866755 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.867019 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.867021 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.867145 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.867213 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.867217 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.868086 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.868090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.868664 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.868673 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.869177 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.869180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.869237 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.869599 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.869610 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.870777 (+ 1167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.872197 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.872199 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.872201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.872408 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.872655 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.872657 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.872659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.872749 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.873112 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.873114 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.873189 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.873274 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.873394 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 10:44:59.873564 10918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.866588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47802 (local address 127.31.237.188:42141)
0808 10:44:59.866742 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.866745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.866757 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:44:59.866824 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.866826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.866858 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.866998 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.867302 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.867910 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.868763 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.869076 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.869157 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.870917 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.870920 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.870922 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.870942 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.872101 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.872490 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.872492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.872493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.872559 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.872829 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.872831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.872832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.873025 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.873099 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.873329 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.873445 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":89}
I20260808 10:44:59.877014 10915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: Bootstrap starting.
I20260808 10:44:59.879455 10915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: Neither blocks nor log segments found. Creating new log.
I20260808 10:44:59.880272 10915 log.cc:824] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: Log is configured to *not* fsync() on all Append() calls
I20260808 10:44:59.882197 10915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: No bootstrap required, opened a new log
I20260808 10:44:59.885334 10915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } }
I20260808 10:44:59.885536 10915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:44:59.885605 10915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56098355de674fc0ae7dbdf27c5dc2e3, State: Initialized, Role: FOLLOWER
I20260808 10:44:59.886049 10915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } }
I20260808 10:44:59.886534 10922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } } }
I20260808 10:44:59.886723 10922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:59.887326 10915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 10:44:59.892465 10946 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:44:59.892550 10946 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:44:59.896018 10657 master_runner.cc:428] Detected that this master 56098355de674fc0ae7dbdf27c5dc2e3 is joining an existing cluster
I20260808 10:44:59.896095 10657 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.237.188:42141
I20260808 10:44:59.908177 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.898947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36088 (local address 127.31.237.190:36401)
0808 10:44:59.899064 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.899068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.899081 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:44:59.899125 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.899128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.899175 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.899258 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.900069 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.900906 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.902493 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.902870 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.902990 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.904582 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.904587 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.904589 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.904615 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.906270 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.906749 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.906753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.906755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.906871 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.907281 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.907283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.907284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.907460 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.907572 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.907894 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.908019 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:44:59.908215 10956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.898422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.0.0.1:36088)
0808 10:44:59.898825 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.898844 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.898931 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.899415 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.899418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.899698 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.899897 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.899907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.901046 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.901049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.902351 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.902362 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.903208 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.903211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.903336 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.903998 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.904038 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.904447 (+ 409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.906384 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.906388 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.906390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.906591 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.906998 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.907003 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.907009 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.907090 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.907562 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.907566 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.907663 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.907792 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.907930 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":178,"spinlock_wait_cycles":1280,"thread_start_us":187,"threads_started":1}
I20260808 10:44:59.908995 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 9846
I20260808 10:44:59.926599 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:32989
--local_ip_for_outbound_sockets=127.31.237.129
--tserver_master_addrs=127.31.237.190:36401
--webserver_port=46675
--webserver_interface=127.31.237.129
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.129.keytab
--principal=oryx/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.237.190:36401,127.31.237.189:35269
--tserver_master_addrs=127.31.237.190:36401,127.31.237.189:35269,127.31.237.188:42141 with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:44:59 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185898, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.190@KRBTEST.COM for oryx/127.31.237.188@KRBTEST.COM
I20260808 10:44:59.927587 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.898447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:35269 (local address 127.0.0.1:59766)
0808 10:44:59.899204 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.899218 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.899348 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.899633 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.899636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.899858 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.900054 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.900064 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.901084 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.901086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.902351 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.902362 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.903142 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.903163 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.903340 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.923135 (+ 19795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.923152 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.923583 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.925716 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.925720 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.925723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.926018 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.926336 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.926339 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.926341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.926439 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.927006 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.927009 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.927105 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.927208 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.927360 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":213,"spinlock_wait_cycles":4480,"thread_start_us":113,"threads_started":1}
I20260808 10:44:59.928165 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.898594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.31.237.189:35269)
0808 10:44:59.898750 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.898755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.899285 (+ 530us) server_negotiation.cc:407] Connection header received
0808 10:44:59.899449 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.899453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.899497 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.899594 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.900191 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.900953 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.902493 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.902870 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.902962 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.923773 (+ 20811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.923778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.923781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.923812 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.925562 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.926146 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.926149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.926152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.926233 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.926568 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.926571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.926572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.926832 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.926951 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.927794 (+ 843us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.927953 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:44:59.932121 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.921414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42141 (local address 127.0.0.1:47826)
0808 10:44:59.921709 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.921726 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.921823 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.922130 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.922135 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.922357 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.922449 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.922502 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.923724 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.923727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.924648 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.924662 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.925345 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.925351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.925490 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.925966 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.925984 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.927537 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.930254 (+ 2717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.930258 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.930261 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.930556 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.930948 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.930956 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.930961 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.931067 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.931586 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.931590 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.931700 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.931783 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.931934 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208}
I20260808 10:44:59.932785 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.921453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47826 (local address 127.31.237.188:42141)
0808 10:44:59.921714 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.921728 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.921772 (+ 44us) server_negotiation.cc:407] Connection header received
0808 10:44:59.921952 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.921955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.922012 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.922147 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.922599 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.923546 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.924812 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.925197 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.927672 (+ 2475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.927800 (+ 128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.927804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.927806 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.927822 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.930124 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.930726 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.930732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.930735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.930828 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.931219 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.931222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.931224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.931456 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.932414 (+ 958us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.932480 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.932604 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":116,"threads_started":1}
I20260808 10:44:59.933362 10918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.899255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47810 (local address 127.31.237.188:42141)
0808 10:44:59.899348 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.899352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.900199 (+ 847us) server_negotiation.cc:407] Connection header received
0808 10:44:59.900380 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.900383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.900422 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.900466 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.901031 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.901812 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.903092 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.903549 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.903629 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.929996 (+ 26367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.929999 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.930002 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.930030 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.931717 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.932165 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.932169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.932173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.932253 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.932543 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.932546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.932548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.932738 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.932811 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.933134 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.933231 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":39}
I20260808 10:44:59.933702 9737 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.237.188:42141: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3"
member_type: NON_VOTER
last_known_addr {
host: "127.31.237.188"
port: 42141
}
attrs {
promote: true
}
}
dest_uuid: "f908020858d84ab8a68a0c90fbc3ea5a"
cas_config_opid_index: 5
I20260808 10:44:59.933812 10959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.898941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42141 (local address 127.0.0.1:47810)
0808 10:44:59.900129 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.900142 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.900278 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.900580 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.900583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.900792 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.900912 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.900920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.901951 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.901954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.902914 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.902982 (+ 68us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.903763 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.903766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.903868 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.929356 (+ 25488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.929372 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.929868 (+ 496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.931824 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.931827 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.931829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.932060 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.932360 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.932366 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.932370 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.932454 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.932887 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.932891 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.932978 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.933529 (+ 551us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.933653 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1128,"thread_start_us":90,"threads_started":1}
I20260808 10:44:59.934412 9737 consensus_queue.cc:237] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: NON_VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } }
I20260808 10:44:59.935704 10636 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 56098355de674fc0ae7dbdf27c5dc2e3 (127.31.237.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: NON_VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } } }
I20260808 10:44:59.936901 10867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:44:59.937232 10636 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.237.188:42141
I20260808 10:44:59.937413 10636 sys_catalog.cc:455] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: NON_VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } } }
I20260808 10:44:59.937532 10636 sys_catalog.cc:458] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:59.937868 10657 master.cc:537] Master@127.31.237.188:42141 shutting down...
I20260808 10:44:59.938719 10867 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 1 FOLLOWER]: Refusing update from remote peer f908020858d84ab8a68a0c90fbc3ea5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 10:44:59.939910 10635 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:44:59.942266 9715 consensus_peers.cc:487] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a -> Peer 56098355de674fc0ae7dbdf27c5dc2e3 (127.31.237.188:42141): Couldn't send request to peer 56098355de674fc0ae7dbdf27c5dc2e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 10:44:59.944644 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.935909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:35269 (local address 127.0.0.1:59778)
0808 10:44:59.936502 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.936516 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.936582 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.938474 (+ 1892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.938481 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.938503 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:44:59.938627 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.938638 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.941378 (+ 2740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.941382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.942573 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.942586 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.944182 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.944188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.944301 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.944353 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.944432 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.944504 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522}
I20260808 10:44:59.945029 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.936135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59778 (local address 127.31.237.189:35269)
0808 10:44:59.937864 (+ 1729us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.937869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.937895 (+ 26us) server_negotiation.cc:407] Connection header received
0808 10:44:59.937943 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.937946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.938001 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.938327 (+ 326us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:44:59.938758 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.940277 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.942741 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.943658 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.943772 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.943922 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.944799 (+ 877us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.944904 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1649}
I20260808 10:44:59.945858 10721 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 LEARNER]: Refusing update from remote peer f908020858d84ab8a68a0c90fbc3ea5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 10:44:59.946303 10966 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: NON_VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 10:44:59.947397 10657 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:44:59.947679 10657 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:44:59.947778 10657 tablet_replica.cc:330] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: stopping tablet replica
I20260808 10:44:59.948503 10721 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 56098355de674fc0ae7dbdf27c5dc2e3 (127.31.237.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: NON_VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } } }
I20260808 10:44:59.950564 10768 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f908020858d84ab8a68a0c90fbc3ea5a. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: NON_VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } } }
I20260808 10:44:59.950755 10768 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: This master's current role is: LEARNER
I20260808 10:44:59.950856 10768 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: NON_VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } } }
I20260808 10:44:59.950943 10768 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: This master's current role is: LEARNER
I20260808 10:44:59.952859 10635 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a: attempting to promote NON_VOTER e2ee44ced7704478a10c8c32f9eb9570 to VOTER
I20260808 10:44:59.953357 10635 consensus_queue.cc:237] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } }
I20260808 10:44:59.956012 10721 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 LEARNER]: Refusing update from remote peer f908020858d84ab8a68a0c90fbc3ea5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 10:44:59.957787 10636 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260808 10:44:59.958294 9715 consensus_peers.cc:487] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a -> Peer 56098355de674fc0ae7dbdf27c5dc2e3 (127.31.237.188:42141): Couldn't send request to peer 56098355de674fc0ae7dbdf27c5dc2e3. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260808 10:44:59.960182 10636 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e2ee44ced7704478a10c8c32f9eb9570 (127.31.237.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } } }
I20260808 10:44:59.960770 10721 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e2ee44ced7704478a10c8c32f9eb9570 (127.31.237.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } } }
I20260808 10:44:59.961678 10636 sys_catalog.cc:455] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } } }
I20260808 10:44:59.961783 10636 sys_catalog.cc:458] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:59.961934 10636 sys_catalog.cc:455] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } } }
I20260808 10:44:59.962039 10636 sys_catalog.cc:458] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: This master's current role is: LEADER
I20260808 10:44:59.962342 10969 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } } }
I20260808 10:44:59.962457 10969 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:44:59.970197 10657 master.cc:552] Master@127.31.237.188:42141 shutdown complete.
I20260808 10:44:59.970266 10657 master_runner.cc:305] Clearing existing system tablet
I20260808 10:44:59.970501 10657 fs_manager.cc:359] Metadata directory not provided
I20260808 10:44:59.970575 10657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:44:59.971766 10657 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:44:59.972637 10977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:44:59.972895 10657 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:44:59.973042 10657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:44:59.973079 10657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/wal
uuid: "56098355de674fc0ae7dbdf27c5dc2e3"
format_stamp: "Formatted at 2026-08-08 10:44:59 on dist-test-slave-twwt"
server_key: "0cdea6d9d5e6c439b59f14f91c126035"
server_key_iv: "d5b4e8e86ba273a3ff8ce4419c492c5b"
server_key_version: "encryptionkey@0"
I20260808 10:44:59.974936 10657 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:44:59.976810 10657 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 10:44:59.976933 10657 log.cc:1196] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 10:44:59.977175 10657 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: Deleting consensus metadata
I20260808 10:44:59.977365 10657 master_runner.cc:315] Copying system tablet from 127.31.237.190:36401
I20260808 10:44:59.978828 10657 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: tablet copy: Beginning tablet copy session from remote peer at address 127.31.237.190:36401
I20260808 10:44:59.987206 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.979793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36098 (local address 127.31.237.190:36401)
0808 10:44:59.979943 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:44:59.979948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:44:59.980132 (+ 184us) server_negotiation.cc:407] Connection header received
0808 10:44:59.980292 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:44:59.980295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:44:59.980347 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:44:59.980438 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:44:59.980908 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.981446 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.982276 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.982498 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.982603 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.983803 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:44:59.983807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:44:59.983810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:44:59.983838 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:44:59.985421 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.985863 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.985866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.985868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.985962 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:44:59.986248 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:44:59.986252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:44:59.986254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:44:59.986444 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:44:59.986554 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:44:59.986926 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 10:44:59.987038 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:44:59.987214 10986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:44:59.979720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.0.0.1:36098)
0808 10:44:59.980026 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 10:44:59.980048 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 10:44:59.980184 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:44:59.980454 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:44:59.980457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:44:59.980677 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:44:59.980781 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.980787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.981558 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.981564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.982145 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:44:59.982153 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:44:59.982641 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:44:59.982645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:44:59.982740 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:44:59.983310 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:44:59.983322 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:44:59.983660 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:44:59.985536 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.985541 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.985544 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.985753 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.986057 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:44:59.986060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:44:59.986062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:44:59.986155 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:44:59.986544 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:44:59.986548 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:44:59.986728 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 10:44:59.986864 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 10:44:59.987016 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":90,"threads_started":1}
I20260808 10:44:59.988018 9767 tablet_copy_service.cc:140] P f908020858d84ab8a68a0c90fbc3ea5a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 56098355de674fc0ae7dbdf27c5dc2e3 ({username='oryx', principal='oryx/127.31.237.188@KRBTEST.COM'} at 127.0.0.1:36098)
I20260808 10:44:59.988144 9767 tablet_copy_service.cc:161] P f908020858d84ab8a68a0c90fbc3ea5a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 56098355de674fc0ae7dbdf27c5dc2e3 at {username='oryx', principal='oryx/127.31.237.188@KRBTEST.COM'} at 127.0.0.1:36098: session id = 56098355de674fc0ae7dbdf27c5dc2e3-00000000000000000000000000000000
I20260808 10:44:59.989181 9767 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 10:44:59.990315 10657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 10:44:59.992317 10657 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: tablet copy: Starting download of 0 data blocks...
I20260808 10:44:59.992528 10657 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: tablet copy: Starting download of 1 WAL segments...
I20260808 10:44:59.993903 10657 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 10:44:59.995520 9767 tablet_copy_service.cc:342] P f908020858d84ab8a68a0c90fbc3ea5a: Request end of tablet copy session 56098355de674fc0ae7dbdf27c5dc2e3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.237.188@KRBTEST.COM'} at 127.0.0.1:36098
I20260808 10:44:59.995616 9767 tablet_copy_service.cc:433] P f908020858d84ab8a68a0c90fbc3ea5a: ending tablet copy session 56098355de674fc0ae7dbdf27c5dc2e3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 56098355de674fc0ae7dbdf27c5dc2e3
I20260808 10:45:00.003598 10657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:00.005702 10987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:00.008019 10992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:00.008229 10657 server_base.cc:1034] running on GCE node
W20260808 10:45:00.011116 10989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:00.011391 10657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:00.011813 10657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:00.012974 10657 hybrid_clock.cc:648] HybridClock initialized: now 1786185900012930 us; error 61 us; skew 500 ppm
Aug 08 10:45:00 dist-test-slave-twwt krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185900, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:00.014899 10657 init.cc:377] Logged in from keytab as oryx/127.31.237.188@KRBTEST.COM (short username oryx)
I20260808 10:45:00.015434 10657 webserver.cc:460] Webserver started at http://127.31.237.188:44039/ using document root <none> and password file <none>
I20260808 10:45:00.015655 10657 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:00.015712 10657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:00.017011 10657 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:45:00.018010 11005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:00.018256 10657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:45:00.018355 10657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/wal
uuid: "56098355de674fc0ae7dbdf27c5dc2e3"
format_stamp: "Formatted at 2026-08-08 10:44:59 on dist-test-slave-twwt"
server_key: "0cdea6d9d5e6c439b59f14f91c126035"
server_key_iv: "d5b4e8e86ba273a3ff8ce4419c492c5b"
server_key_version: "encryptionkey@0"
I20260808 10:45:00.018471 10657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:00.025367 10657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:00.027599 10657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:00.032779 10657 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.188:42141
I20260808 10:45:00.033221 11093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.188:42141 every 8 connection(s)
I20260808 10:45:00.033576 10657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/master-2/data/info.pb
I20260808 10:45:00.035369 11094 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:45:00.036536 11094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: Bootstrap starting.
I20260808 10:45:00.046432 11094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:45:00.046779 11094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: Bootstrap complete.
I20260808 10:45:00.047284 11094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } }
I20260808 10:45:00.047432 11094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 56098355de674fc0ae7dbdf27c5dc2e3, State: Initialized, Role: LEARNER
I20260808 10:45:00.047559 11094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } }
I20260808 10:45:00.047904 11102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } } }
I20260808 10:45:00.048077 11102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [sys.catalog]: This master's current role is: LEARNER
I20260808 10:45:00.048090 11094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:45:00.052104 11121 catalog_manager.cc:1259] Loaded cluster ID: 282a68a2c33345ef84eece0d29bd333f
I20260808 10:45:00.052181 11121 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: loading cluster ID for follower catalog manager: success
I20260808 10:45:00.053817 11121 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: acquiring CA information for follower catalog manager: success
I20260808 10:45:00.055368 11121 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:45:00.063153 10275 heartbeater.cc:503] Master 127.31.237.190:36401 was elected leader, sending a full tablet report...
W20260808 10:45:00.097481 10963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:00.097828 10963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:00.097898 10963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:00.104609 10963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:00.104709 10963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:00.104877 10963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:45:00.113121 10963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.129.keytab
--principal=oryx/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:32989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=46675
--tserver_master_addrs=127.31.237.190:36401,127.31.237.189:35269,127.31.237.188:42141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:00.114346 10963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:00.115772 10963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:00.126688 11130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:00.126775 11137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:00.126914 10963 server_base.cc:1034] running on GCE node
W20260808 10:45:00.126690 11131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:00.127487 10963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:00.128166 10963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:00.129354 10963 hybrid_clock.cc:648] HybridClock initialized: now 1786185900129337 us; error 29 us; skew 500 ppm
Aug 08 10:45:00 dist-test-slave-twwt krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185900, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:00.132313 10963 init.cc:377] Logged in from keytab as oryx/127.31.237.129@KRBTEST.COM (short username oryx)
I20260808 10:45:00.133180 10963 webserver.cc:460] Webserver started at http://127.31.237.129:46675/ using document root <none> and password file <none>
I20260808 10:45:00.133709 10963 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:00.133780 10963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:00.137779 10963 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:45:00.139997 11149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:00.141390 10963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:45:00.141635 10963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "8cff3352097c4b25a3805d484ad98c80"
format_stamp: "Formatted at 2026-08-08 10:44:58 on dist-test-slave-twwt"
server_key: "f1f7a4186aa0604f9d8036f11f542d6d"
server_key_iv: "8df7198ecdf0b93980050e5b4ae60cdc"
server_key_version: "encryptionkey@0"
I20260808 10:45:00.142186 10963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:00.168773 10963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:00.172281 10963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:00.172502 10963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:00.173166 10963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:00.174110 10963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:00.174185 10963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:00.174259 10963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:00.174305 10963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:00.185081 10963 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:32989
I20260808 10:45:00.185105 11314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:32989 every 8 connection(s)
I20260808 10:45:00.186129 10963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:45:00.187392 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 10963
I20260808 10:45:00.187605 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 10073
I20260808 10:45:00.199307 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:35331
--local_ip_for_outbound_sockets=127.31.237.130
--tserver_master_addrs=127.31.237.190:36401
--webserver_port=37611
--webserver_interface=127.31.237.130
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.130.keytab
--principal=oryx/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.237.190:36401,127.31.237.189:35269
--tserver_master_addrs=127.31.237.190:36401,127.31.237.189:35269,127.31.237.188:42141 with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:45:00 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185900, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.189@KRBTEST.COM
Aug 08 10:45:00 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185900, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.188@KRBTEST.COM
Aug 08 10:45:00 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185900, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
I20260808 10:45:00.207693 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.188410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:59849 (local address 127.31.237.190:36401)
0808 10:45:00.188606 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.188613 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.192032 (+ 3419us) server_negotiation.cc:407] Connection header received
0808 10:45:00.193261 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.193265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.193322 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.193386 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:00.195593 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.196487 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.198472 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.198884 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.198970 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.203923 (+ 4953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:00.203928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:00.203931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:00.203967 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:00.205883 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.206354 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.206358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.206359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.206441 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.206790 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.206794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.206795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.207002 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:00.207090 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.207420 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.207538 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 10:45:00.207857 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.192674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:46219 (local address 127.31.237.188:42141)
0808 10:45:00.192986 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.192993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.193012 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:45:00.193356 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.193363 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.193435 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.193526 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:00.195569 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.196647 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.198476 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.198918 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.199011 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.203195 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:00.203198 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:00.203200 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:00.203226 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:00.205403 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.205958 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.205964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.205968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.206068 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.206437 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.206442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.206447 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.206758 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:00.206866 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.207446 (+ 580us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.207601 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":105,"threads_started":1}
I20260808 10:45:00.208703 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.191110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.129:59849)
0808 10:45:00.191952 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.191964 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.192995 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.193725 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.193734 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.194489 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:00.195385 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.195409 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.196646 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.196652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.198337 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.198349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.199174 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.199180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.199763 (+ 583us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.200769 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:00.200790 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:00.203751 (+ 2961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:00.205995 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.205998 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.205999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.206241 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.206553 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.206558 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.206560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.206670 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.207148 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:00.207152 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:00.207230 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.208374 (+ 1144us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.208514 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"mutex_wait_us":790,"spinlock_wait_cycles":19968,"thread_start_us":97,"threads_started":1}
I20260808 10:45:00.208949 11320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.188618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:35269 (local address 127.31.237.129:50803)
0808 10:45:00.191701 (+ 3083us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.191761 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.192995 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.193725 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.193734 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.194489 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:00.195392 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.195410 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.196574 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.196577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.198330 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.198336 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.199174 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.199178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.199510 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.200424 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:00.200435 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:00.202288 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:00.205119 (+ 2831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.205129 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.205151 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.205534 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.205875 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.205878 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.205880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.206062 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.206580 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:00.206588 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:00.206935 (+ 347us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.207138 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.207512 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2851,"mutex_wait_us":533,"spinlock_wait_cycles":11008,"thread_start_us":1970,"threads_started":1}
I20260808 10:45:00.209641 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.190922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:50803 (local address 127.31.237.189:35269)
0808 10:45:00.191042 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.191046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.191879 (+ 833us) server_negotiation.cc:407] Connection header received
0808 10:45:00.193260 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.193264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.193322 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.193380 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:00.195565 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.196398 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.198469 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.198854 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.198975 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.202475 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:00.202482 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:00.202488 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:00.202523 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:00.204977 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.205668 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.205672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.205674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.205763 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.206178 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.206181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.206183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.206436 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:00.206530 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.209337 (+ 2807us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.209467 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:45:00.210085 11321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.191632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42141 (local address 127.31.237.129:46219)
0808 10:45:00.192386 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.192399 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.193059 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.193726 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.193736 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.194494 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:00.195393 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.195410 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.196773 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.196777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.198319 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.198331 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.199174 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.199178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.199511 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.200376 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:00.200403 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:00.202750 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:00.205540 (+ 2790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.205546 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.205550 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.205829 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.206189 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.206195 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.206199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.206300 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.206924 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:00.206929 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:00.207048 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.207168 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.209920 (+ 2752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":3296,"spinlock_wait_cycles":23168,"thread_start_us":69,"threads_started":1}
I20260808 10:45:00.210245 11315 heartbeater.cc:347] Connected to a master server at 127.31.237.190:36401
I20260808 10:45:00.210542 11315 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:00.210965 11316 heartbeater.cc:347] Connected to a master server at 127.31.237.188:42141
I20260808 10:45:00.211053 11316 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:00.211234 11315 heartbeater.cc:511] Master 127.31.237.190:36401 requested a full tablet report, sending...
I20260808 10:45:00.211248 11316 heartbeater.cc:511] Master 127.31.237.188:42141 requested a full tablet report, sending...
I20260808 10:45:00.212026 11317 heartbeater.cc:347] Connected to a master server at 127.31.237.189:35269
I20260808 10:45:00.212126 11317 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:00.212321 11317 heartbeater.cc:511] Master 127.31.237.189:35269 requested a full tablet report, sending...
I20260808 10:45:00.213367 9737 ts_manager.cc:194] Re-registered known tserver with Master: 8cff3352097c4b25a3805d484ad98c80 (127.31.237.129:32989)
I20260808 10:45:00.213675 10705 ts_manager.cc:194] Registered new tserver with Master: 8cff3352097c4b25a3805d484ad98c80 (127.31.237.129:32989)
I20260808 10:45:00.213768 11034 ts_manager.cc:194] Registered new tserver with Master: 8cff3352097c4b25a3805d484ad98c80 (127.31.237.129:32989)
I20260808 10:45:00.214164 9737 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:59849
I20260808 10:45:00.285948 10480 heartbeater.cc:503] Master 127.31.237.190:36401 was elected leader, sending a full tablet report...
W20260808 10:45:00.323539 11327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:00.323820 11327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:00.323874 11327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:00.327754 11327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:00.327803 11327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:00.327864 11327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:45:00.332850 11327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.130.keytab
--principal=oryx/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:35331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=37611
--tserver_master_addrs=127.31.237.190:36401,127.31.237.189:35269,127.31.237.188:42141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:00.333878 11327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:00.334944 11327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:00.344633 11337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:00.344650 11335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:00.344650 11334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:00.344705 11327 server_base.cc:1034] running on GCE node
I20260808 10:45:00.345309 11327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:00.345983 11327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:00.347141 11327 hybrid_clock.cc:648] HybridClock initialized: now 1786185900347105 us; error 46 us; skew 500 ppm
Aug 08 10:45:00 dist-test-slave-twwt krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185900, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:00.350073 11327 init.cc:377] Logged in from keytab as oryx/127.31.237.130@KRBTEST.COM (short username oryx)
I20260808 10:45:00.351023 11327 webserver.cc:460] Webserver started at http://127.31.237.130:37611/ using document root <none> and password file <none>
I20260808 10:45:00.351580 11327 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:00.351683 11327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:00.355643 11327 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:45:00.357694 11345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:00.359050 11327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:45:00.359202 11327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "7910b12be2c048c1bde2a069315e9c32"
format_stamp: "Formatted at 2026-08-08 10:44:58 on dist-test-slave-twwt"
server_key: "f603b93946819bbe72f90287259fcf87"
server_key_iv: "981a213b8752cb1f7ec0a5dd884ff97f"
server_key_version: "encryptionkey@0"
I20260808 10:45:00.359855 11327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:00.370043 11327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:00.372951 11327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:00.373162 11327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:00.373775 11327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:00.374744 11327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:00.374811 11327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:00.374888 11327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:00.374930 11327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:00.385181 11327 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:35331
I20260808 10:45:00.385203 11522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:35331 every 8 connection(s)
I20260808 10:45:00.386227 11327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:45:00.388715 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 11327
I20260808 10:45:00.388916 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 10282
Aug 08 10:45:00 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185900, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.188@KRBTEST.COM
Aug 08 10:45:00 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185900, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
Aug 08 10:45:00 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185900, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.189@KRBTEST.COM
I20260808 10:45:00.411381 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:45425
--local_ip_for_outbound_sockets=127.31.237.131
--tserver_master_addrs=127.31.237.190:36401
--webserver_port=35701
--webserver_interface=127.31.237.131
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.131.keytab
--principal=oryx/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.237.190:36401,127.31.237.189:35269
--tserver_master_addrs=127.31.237.190:36401,127.31.237.189:35269,127.31.237.188:42141 with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:45:00.413545 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.388302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:54335 (local address 127.31.237.188:42141)
0808 10:45:00.388453 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.388459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.397477 (+ 9018us) server_negotiation.cc:407] Connection header received
0808 10:45:00.399508 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.399515 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.399594 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.399654 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:00.401956 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.402734 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.404563 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.404935 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.405007 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.408655 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:00.408662 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:00.408668 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:00.408701 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:00.410583 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.411316 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.411322 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.411327 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.411443 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.411944 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.411951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.411956 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.412204 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:00.412287 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.413209 (+ 922us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.413362 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:45:00.413734 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.392959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:47849 (local address 127.31.237.190:36401)
0808 10:45:00.393133 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.393139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.403288 (+ 10149us) server_negotiation.cc:407] Connection header received
0808 10:45:00.403498 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.403502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.403558 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.403611 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:00.404241 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.405029 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.406196 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.406543 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.406653 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.409909 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:00.409913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:00.409918 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:00.409957 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:00.411890 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.412379 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.412382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.412384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.412476 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.412819 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.412823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.412824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.413040 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:00.413113 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.413476 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.413593 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:45:00.415549 11539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.390029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.130:47849)
0808 10:45:00.403185 (+ 13156us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.403200 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.403376 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.403767 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.403770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.403969 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:00.404089 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.404096 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.405167 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.405169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.406045 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.406061 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.406829 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.406835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.406978 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.407542 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:00.407560 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:00.409727 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:00.412002 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.412007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.412009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.412232 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.412592 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.412595 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.412597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.412683 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.413187 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:00.413190 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:00.413272 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.413939 (+ 667us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.414356 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13072,"thread_start_us":1270,"threads_started":1}
I20260808 10:45:00.415668 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.388302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:41931 (local address 127.31.237.189:35269)
0808 10:45:00.388454 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.388458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.402992 (+ 14534us) server_negotiation.cc:407] Connection header received
0808 10:45:00.403200 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.403207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.403275 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.403335 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:00.404213 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.405118 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.406476 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.406902 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.406995 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.409944 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:00.409951 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:00.409957 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:00.409995 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:00.412077 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.414263 (+ 2186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.414269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.414273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.414366 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.414744 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.414749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.414752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.415000 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:00.415077 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.415394 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.415507 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 10:45:00.416033 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.388917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:35269 (local address 127.31.237.130:41931)
0808 10:45:00.402873 (+ 13956us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.402893 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.403082 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.403529 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.403535 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.403795 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:00.404055 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.404068 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.405274 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.405280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.406321 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.406336 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.407238 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.407245 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.407392 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.407945 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:00.407965 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:00.409744 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:00.413856 (+ 4112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.413861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.413865 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.414137 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.414504 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.414507 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.414509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.414625 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.415154 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:00.415156 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:00.415226 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.415339 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.415495 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13832,"mutex_wait_us":31,"spinlock_wait_cycles":16768,"thread_start_us":147,"threads_started":1}
I20260808 10:45:00.417344 11531 heartbeater.cc:347] Connected to a master server at 127.31.237.189:35269
I20260808 10:45:00.417634 11531 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:00.417685 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.388917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42141 (local address 127.31.237.130:54335)
0808 10:45:00.397202 (+ 8285us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.397317 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.399146 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.400237 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.400254 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.401051 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:00.401751 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.401772 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.402921 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.402928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.404419 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.404429 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.405234 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.405238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.405447 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.406178 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:00.406201 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:00.408444 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:00.410754 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.410762 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.410783 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.411181 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.411594 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.411600 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.411604 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.411806 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.412374 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:00.412381 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:00.412875 (+ 494us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.417380 (+ 4505us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.417527 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7926,"thread_start_us":133,"threads_started":1}
I20260808 10:45:00.417344 11528 heartbeater.cc:347] Connected to a master server at 127.31.237.190:36401
I20260808 10:45:00.418201 11528 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:00.418282 11531 heartbeater.cc:511] Master 127.31.237.189:35269 requested a full tablet report, sending...
I20260808 10:45:00.418339 11528 heartbeater.cc:511] Master 127.31.237.190:36401 requested a full tablet report, sending...
I20260808 10:45:00.419056 11530 heartbeater.cc:347] Connected to a master server at 127.31.237.188:42141
I20260808 10:45:00.419144 11530 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:00.419314 11530 heartbeater.cc:511] Master 127.31.237.188:42141 requested a full tablet report, sending...
I20260808 10:45:00.419549 9737 ts_manager.cc:194] Re-registered known tserver with Master: 7910b12be2c048c1bde2a069315e9c32 (127.31.237.130:35331)
I20260808 10:45:00.419590 10705 ts_manager.cc:194] Registered new tserver with Master: 7910b12be2c048c1bde2a069315e9c32 (127.31.237.130:35331)
I20260808 10:45:00.420557 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.412888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42141 (local address 127.0.0.1:47834)
0808 10:45:00.413194 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.413211 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.413325 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.414638 (+ 1313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.414642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.414662 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:00.414749 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.414756 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.416089 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.416095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.417386 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.417402 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.420086 (+ 2684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.420090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.420214 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.420238 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.420311 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.420389 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":3}
I20260808 10:45:00.420840 11034 ts_manager.cc:194] Registered new tserver with Master: 7910b12be2c048c1bde2a069315e9c32 (127.31.237.130:35331)
I20260808 10:45:00.420941 9737 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:47849
I20260808 10:45:00.421058 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.413504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.31.237.188:42141)
0808 10:45:00.413712 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.413716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.413728 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:45:00.413771 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.413774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.413835 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.413962 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:00.414892 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.415587 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.417562 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.419696 (+ 2134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.419820 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.420014 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.420775 (+ 761us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.420851 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":73,"threads_started":1}
I20260808 10:45:00.421650 11054 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 10:45:00.422185 11102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f908020858d84ab8a68a0c90fbc3ea5a. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } } }
I20260808 10:45:00.422312 11102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [sys.catalog]: This master's current role is: LEARNER
W20260808 10:45:00.536695 11541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:00.536975 11541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:00.537019 11541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:00.540959 11541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:00.541047 11541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:00.541143 11541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:45:00.546813 11541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.131.keytab
--principal=oryx/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:45425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=35701
--tserver_master_addrs=127.31.237.190:36401,127.31.237.189:35269,127.31.237.188:42141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:00.547926 11541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:00.549295 11541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:00.559485 11551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:00.559546 11541 server_base.cc:1034] running on GCE node
W20260808 10:45:00.559507 11554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:00.559514 11548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:00.560216 11541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:00.560878 11541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:00.562070 11541 hybrid_clock.cc:648] HybridClock initialized: now 1786185900562035 us; error 44 us; skew 500 ppm
Aug 08 10:45:00 dist-test-slave-twwt krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185900, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:00.571709 11541 init.cc:377] Logged in from keytab as oryx/127.31.237.131@KRBTEST.COM (short username oryx)
I20260808 10:45:00.572697 11541 webserver.cc:460] Webserver started at http://127.31.237.131:35701/ using document root <none> and password file <none>
I20260808 10:45:00.573324 11541 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:00.573431 11541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:00.577692 11541 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:45:00.580011 11561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:00.581374 11541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:45:00.581511 11541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "11f5c1015791423bbf99231d8cc6f903"
format_stamp: "Formatted at 2026-08-08 10:44:59 on dist-test-slave-twwt"
server_key: "82877612b4e0534263c1301fa3b2fb2e"
server_key_iv: "0b8e464b0567dc50e6e08a5140dc5f61"
server_key_version: "encryptionkey@0"
I20260808 10:45:00.582100 11541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:00.601473 11541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:00.604365 11541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:00.604578 11541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:00.605224 11541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:00.606105 11541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:00.606153 11541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:00.606220 11541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:00.606252 11541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:00.615511 11541 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:45425
I20260808 10:45:00.615572 11728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:45425 every 8 connection(s)
I20260808 10:45:00.616621 11541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:45:00.621170 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 11541
Aug 08 10:45:00 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185900, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
Aug 08 10:45:00 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185900, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.189@KRBTEST.COM
Aug 08 10:45:00 dist-test-slave-twwt krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185900, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.188@KRBTEST.COM
I20260808 10:45:00.636572 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.620357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:35535 (local address 127.31.237.190:36401)
0808 10:45:00.620501 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.620505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.620608 (+ 103us) server_negotiation.cc:407] Connection header received
0808 10:45:00.621198 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.621202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.621257 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.621312 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:00.623227 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.624058 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.625531 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.625866 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.625951 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.630786 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:00.630792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:00.630799 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:00.630831 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:00.633492 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.634288 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.634295 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.634299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.634417 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.634942 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.634946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.634948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.635193 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:00.635281 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.636185 (+ 904us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.636338 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:45:00.638413 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.624160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47844 (local address 127.31.237.188:42141)
0808 10:45:00.624284 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.624289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.625999 (+ 1710us) server_negotiation.cc:407] Connection header received
0808 10:45:00.626193 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.626197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.626259 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.626325 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:00.627901 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.628689 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.630290 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.630739 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.630835 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.632858 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:00.632865 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:00.632870 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:00.632903 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:00.635513 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.636953 (+ 1440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.636956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.636959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.637056 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.637400 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.637403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.637405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.637628 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:00.637736 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.638138 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.638272 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:45:00.638651 11739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.620218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.131:35535)
0808 10:45:00.620516 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.620528 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.620951 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.621742 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.621751 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.622367 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:00.623083 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.623099 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.624439 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.624442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.625392 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.625403 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.626149 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.626153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.626346 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.627690 (+ 1344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:00.627703 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:00.630607 (+ 2904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:00.633627 (+ 3020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.633636 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.633660 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.634156 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.634554 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.634559 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.634561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.634821 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.635393 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:00.635403 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:00.635842 (+ 439us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.636861 (+ 1019us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.637382 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":62,"thread_start_us":84,"threads_started":1}
I20260808 10:45:00.639151 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.618726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:34311 (local address 127.31.237.189:35269)
0808 10:45:00.618858 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.618862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.619780 (+ 918us) server_negotiation.cc:407] Connection header received
0808 10:45:00.621212 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.621217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.621284 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.621344 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:00.624436 (+ 3092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.625221 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.626314 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.626715 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.626811 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.630981 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:00.630985 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:00.630987 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:00.631016 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:00.635547 (+ 4531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.636072 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.636078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.636082 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.636181 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.636557 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.636564 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.636567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.636940 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:00.637032 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.638877 (+ 1845us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.639003 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:45:00.639369 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.619015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:35269 (local address 127.31.237.131:34311)
0808 10:45:00.619604 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.619660 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.620943 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.621742 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.621751 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.624193 (+ 2442us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:00.624302 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.624309 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.625351 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.625354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.626188 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.626196 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.626925 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.626929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.627029 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.627523 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:00.627546 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:00.630163 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:00.635682 (+ 5519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.635687 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.635692 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.635962 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.636306 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.636310 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.636311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.636443 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.637082 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:00.637088 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:00.637273 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.637393 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.637546 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":3,"thread_start_us":680,"threads_started":1}
I20260808 10:45:00.640283 11729 heartbeater.cc:347] Connected to a master server at 127.31.237.190:36401
I20260808 10:45:00.640331 11733 heartbeater.cc:347] Connected to a master server at 127.31.237.189:35269
I20260808 10:45:00.640547 11733 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:00.640544 11729 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:00.641222 11729 heartbeater.cc:511] Master 127.31.237.190:36401 requested a full tablet report, sending...
I20260808 10:45:00.641222 11733 heartbeater.cc:511] Master 127.31.237.189:35269 requested a full tablet report, sending...
I20260808 10:45:00.642513 10705 ts_manager.cc:194] Registered new tserver with Master: 11f5c1015791423bbf99231d8cc6f903 (127.31.237.131:45425)
I20260808 10:45:00.642926 11746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.623393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36114 (local address 127.31.237.190:36401)
0808 10:45:00.625126 (+ 1733us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.625130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.625142 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:45:00.625177 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.625180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.625224 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.625316 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:00.628941 (+ 3625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.629782 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.630820 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.631220 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.631330 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.633132 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:00.633138 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:00.633140 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:00.633168 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:00.640828 (+ 7660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.641345 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.641349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.641350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.641443 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.641758 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.641762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.641763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.642013 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:00.642136 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.642460 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.642740 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1659,"thread_start_us":65,"threads_started":1}
I20260808 10:45:00.642993 9737 ts_manager.cc:194] Re-registered known tserver with Master: 11f5c1015791423bbf99231d8cc6f903 (127.31.237.131:45425)
I20260808 10:45:00.643630 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.618473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:44131 (local address 127.31.237.188:42141)
0808 10:45:00.618635 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.618641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.620055 (+ 1414us) server_negotiation.cc:407] Connection header received
0808 10:45:00.621199 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.621203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.621261 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.621314 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:00.623554 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.624294 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.631195 (+ 6901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.631536 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.631619 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.634227 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:00.634233 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:00.634239 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:00.634273 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:00.637452 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.641583 (+ 4131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.641586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.641588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.641677 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.642013 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.642016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.642018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.642237 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:00.642325 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.643325 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.643445 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:45:00.643935 9737 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:35535
I20260808 10:45:00.644363 11738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.619015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42141 (local address 127.31.237.131:44131)
0808 10:45:00.619956 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.619970 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.620951 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.621742 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.621751 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.623265 (+ 1514us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:00.623417 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.623425 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.626705 (+ 3280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.626708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.627533 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.627544 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.631827 (+ 4283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.631830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.631974 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.632403 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:00.632422 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:00.634069 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:00.641202 (+ 7133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.641205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.641208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.641456 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.641788 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:00.641790 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:00.641792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:00.641897 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:00.642385 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:00.642388 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:00.642476 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.644021 (+ 1545us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.644159 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"mutex_wait_us":44,"thread_start_us":162,"threads_started":1}
I20260808 10:45:00.645061 11730 heartbeater.cc:347] Connected to a master server at 127.31.237.188:42141
I20260808 10:45:00.645174 11730 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:00.645328 11730 heartbeater.cc:511] Master 127.31.237.188:42141 requested a full tablet report, sending...
I20260808 10:45:00.646052 11034 ts_manager.cc:194] Registered new tserver with Master: 11f5c1015791423bbf99231d8cc6f903 (127.31.237.131:45425)
I20260808 10:45:00.646486 11749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.623491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59788 (local address 127.31.237.189:35269)
0808 10:45:00.625507 (+ 2016us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.625510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.625520 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:45:00.625548 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.625549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.625586 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.625689 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:00.626712 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.627590 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.628829 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.629194 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.629274 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.634975 (+ 5701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:00.634981 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:00.634986 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:00.635017 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:00.641099 (+ 6082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.644568 (+ 3469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.644572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.644574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.644667 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:00.645151 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:00.645155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:00.645157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:00.645372 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:00.645497 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.646122 (+ 625us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.646277 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1953,"thread_start_us":67,"threads_started":1}
I20260808 10:45:00.647511 9737 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:45:00.650842 9737 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:45:00.653489 10636 sys_catalog.cc:455] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: NON_VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: true } } }
I20260808 10:45:00.653622 10636 sys_catalog.cc:458] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:00.657488 11758 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a: attempting to promote NON_VOTER 56098355de674fc0ae7dbdf27c5dc2e3 to VOTER
I20260808 10:45:00.657846 11758 consensus_queue.cc:237] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } }
I20260808 10:45:00.658981 11054 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 1 LEARNER]: Refusing update from remote peer f908020858d84ab8a68a0c90fbc3ea5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 10:45:00.659373 11757 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 10:45:00.661017 10636 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 56098355de674fc0ae7dbdf27c5dc2e3 (127.31.237.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } } }
I20260808 10:45:00.661612 10721 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 FOLLOWER]: Refusing update from remote peer f908020858d84ab8a68a0c90fbc3ea5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 10:45:00.661463 11054 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 56098355de674fc0ae7dbdf27c5dc2e3 (127.31.237.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } } }
I20260808 10:45:00.662047 11758 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 10:45:00.662482 11758 sys_catalog.cc:455] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } } }
I20260808 10:45:00.662577 11758 sys_catalog.cc:458] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:00.662513 10636 sys_catalog.cc:455] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } } }
I20260808 10:45:00.662689 10636 sys_catalog.cc:458] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:00.663448 10721 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 56098355de674fc0ae7dbdf27c5dc2e3 (127.31.237.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } } }
I20260808 10:45:00.663617 11102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } } }
I20260808 10:45:00.663722 11102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:45:00.665123 11772 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } } }
I20260808 10:45:00.665300 11772 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:45:00.665657 11758 sys_catalog.cc:455] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } } }
I20260808 10:45:00.665784 11758 sys_catalog.cc:458] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:00.674208 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.668196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:45425 (local address 127.0.0.1:53708)
0808 10:45:00.668372 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.668407 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.668488 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.669567 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.669570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.669590 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:00.669726 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.669734 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.671281 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.671287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.672377 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.672388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.673772 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.673776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.673895 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.673917 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.674001 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.674073 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":5}
I20260808 10:45:00.674703 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.669554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:35331 (local address 127.0.0.1:43604)
0808 10:45:00.670346 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.670360 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.670456 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.670944 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.670947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.670965 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:00.671045 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.671050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.672194 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.672199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.673255 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.673266 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.674288 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.674292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.674375 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.674398 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.674470 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.674544 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":90,"threads_started":1}
I20260808 10:45:00.674755 11783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.669973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43604 (local address 127.31.237.130:35331)
0808 10:45:00.670309 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.670321 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.670451 (+ 130us) server_negotiation.cc:407] Connection header received
0808 10:45:00.670560 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.670565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.670799 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.670951 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:00.671137 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.672081 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.673371 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.674159 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.674291 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.674420 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.674506 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.674595 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":109,"threads_started":1}
I20260808 10:45:00.674893 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.668543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:32989 (local address 127.0.0.1:45838)
0808 10:45:00.668829 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.668858 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.668941 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.669959 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.669964 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.669986 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:00.670083 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.670088 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.671666 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.671669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.672703 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.672711 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.673775 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.673778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.674515 (+ 737us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.674544 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.674651 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.674743 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":606,"thread_start_us":150,"threads_started":1}
I20260808 10:45:00.675268 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.669106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45838 (local address 127.31.237.129:32989)
0808 10:45:00.669409 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.669427 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.669459 (+ 32us) server_negotiation.cc:407] Connection header received
0808 10:45:00.669543 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.669548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.669774 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.670259 (+ 485us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:00.670425 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.671540 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.672829 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.673640 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.673910 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.674038 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.675007 (+ 969us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.675113 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":85,"threads_started":1}
I20260808 10:45:00.674998 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.668674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53708 (local address 127.31.237.131:45425)
0808 10:45:00.669007 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.669020 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.669046 (+ 26us) server_negotiation.cc:407] Connection header received
0808 10:45:00.669128 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.669134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.669395 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.669903 (+ 508us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:00.670056 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.671173 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.672889 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.673638 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.673774 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.673924 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.674033 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.674130 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":139,"threads_started":1}
I20260808 10:45:00.678077 11424 tablet_service.cc:1467] Processing CreateTablet for tablet 3ef1c057742e4127919d5fd712883169 (DEFAULT_TABLE table=test-table [id=90b4b6228a39426b9e72d63098386504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:00.678125 11218 tablet_service.cc:1467] Processing CreateTablet for tablet 3ef1c057742e4127919d5fd712883169 (DEFAULT_TABLE table=test-table [id=90b4b6228a39426b9e72d63098386504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:00.678567 11631 tablet_service.cc:1467] Processing CreateTablet for tablet 3ef1c057742e4127919d5fd712883169 (DEFAULT_TABLE table=test-table [id=90b4b6228a39426b9e72d63098386504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:00.679226 11218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ef1c057742e4127919d5fd712883169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:00.679308 11424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ef1c057742e4127919d5fd712883169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:00.679512 11631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ef1c057742e4127919d5fd712883169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:00.688059 11786 tablet_bootstrap.cc:492] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32: Bootstrap starting.
I20260808 10:45:00.688845 11788 tablet_bootstrap.cc:492] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80: Bootstrap starting.
I20260808 10:45:00.690685 11786 tablet_bootstrap.cc:654] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:00.690685 11788 tablet_bootstrap.cc:654] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:00.691068 11789 tablet_bootstrap.cc:492] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903: Bootstrap starting.
I20260808 10:45:00.692140 11786 log.cc:824] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:00.692252 11788 log.cc:824] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:00.693611 11789 tablet_bootstrap.cc:654] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:00.694480 11789 log.cc:824] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:00.694556 11786 tablet_bootstrap.cc:492] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32: No bootstrap required, opened a new log
I20260808 10:45:00.694787 11786 ts_tablet_manager.cc:1397] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 10:45:00.695415 11788 tablet_bootstrap.cc:492] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80: No bootstrap required, opened a new log
I20260808 10:45:00.695665 11788 ts_tablet_manager.cc:1397] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260808 10:45:00.697145 11789 tablet_bootstrap.cc:492] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903: No bootstrap required, opened a new log
I20260808 10:45:00.697357 11789 ts_tablet_manager.cc:1397] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260808 10:45:00.701084 11788 raft_consensus.cc:348] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:00.701334 11788 raft_consensus.cc:374] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:00.701396 11788 raft_consensus.cc:729] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cff3352097c4b25a3805d484ad98c80, State: Initialized, Role: FOLLOWER
I20260808 10:45:00.701208 11786 raft_consensus.cc:348] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:00.701478 11786 raft_consensus.cc:374] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:00.701538 11786 raft_consensus.cc:729] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7910b12be2c048c1bde2a069315e9c32, State: Initialized, Role: FOLLOWER
I20260808 10:45:00.701862 11788 consensus_queue.cc:260] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:00.701879 11789 raft_consensus.cc:348] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:00.702101 11789 raft_consensus.cc:374] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:00.702056 11786 consensus_queue.cc:260] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:00.702143 11789 raft_consensus.cc:729] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f5c1015791423bbf99231d8cc6f903, State: Initialized, Role: FOLLOWER
I20260808 10:45:00.702553 11789 consensus_queue.cc:260] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:00.703244 11786 ts_tablet_manager.cc:1428] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32: Time spent starting tablet: real 0.008s user 0.004s sys 0.005s
I20260808 10:45:00.703647 11528 heartbeater.cc:503] Master 127.31.237.190:36401 was elected leader, sending a full tablet report...
I20260808 10:45:00.703775 11788 ts_tablet_manager.cc:1428] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80: Time spent starting tablet: real 0.008s user 0.002s sys 0.004s
I20260808 10:45:00.703794 11789 ts_tablet_manager.cc:1428] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:45:00.704131 11315 heartbeater.cc:503] Master 127.31.237.190:36401 was elected leader, sending a full tablet report...
I20260808 10:45:00.707448 11729 heartbeater.cc:503] Master 127.31.237.190:36401 was elected leader, sending a full tablet report...
W20260808 10:45:00.868891 11735 tablet.cc:2369] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:45:00.888566 11532 tablet.cc:2369] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:45:00.926949 11794 raft_consensus.cc:482] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:45:00.927229 11794 raft_consensus.cc:504] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:00.928525 11794 leader_election.cc:288] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11f5c1015791423bbf99231d8cc6f903 (127.31.237.131:45425), 8cff3352097c4b25a3805d484ad98c80 (127.31.237.129:32989)
I20260808 10:45:00.934254 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.929048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:46099 (local address 127.31.237.131:45425)
0808 10:45:00.929163 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.929168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.929435 (+ 267us) server_negotiation.cc:407] Connection header received
0808 10:45:00.929600 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.929603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.929660 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.929758 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:00.930154 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.930988 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.932838 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.933451 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.933586 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.933650 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.934035 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.934115 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:45:00.934254 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.929013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:55721 (local address 127.31.237.129:32989)
0808 10:45:00.929156 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.929162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.929436 (+ 274us) server_negotiation.cc:407] Connection header received
0808 10:45:00.929599 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.929603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.929660 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.929758 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:00.930154 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.930981 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.932838 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.933456 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.933586 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.933645 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.934035 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.934115 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:45:00.934303 11798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.928879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:32989 (local address 127.31.237.130:55721)
0808 10:45:00.929330 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.929342 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.929475 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.929790 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.929816 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.929842 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:00.930010 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.930018 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.931126 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.931130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.932674 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.932687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.933590 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.933594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.933851 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.933910 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.933996 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.934110 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":335,"spinlock_wait_cycles":11264,"thread_start_us":64,"threads_started":1}
I20260808 10:45:00.934312 11797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.928879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:45425 (local address 127.31.237.130:46099)
0808 10:45:00.929315 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:00.929331 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:00.929475 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:00.929842 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:00.929844 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:00.929856 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:00.930030 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.930035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.931126 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.931130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.932674 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:00.932687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.933590 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.933594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:00.933878 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.933915 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 10:45:00.933996 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 10:45:00.934110 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":241,"spinlock_wait_cycles":14976,"thread_start_us":87,"threads_started":1}
I20260808 10:45:00.935148 11247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ef1c057742e4127919d5fd712883169" candidate_uuid: "7910b12be2c048c1bde2a069315e9c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cff3352097c4b25a3805d484ad98c80" is_pre_election: true
I20260808 10:45:00.935304 11660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ef1c057742e4127919d5fd712883169" candidate_uuid: "7910b12be2c048c1bde2a069315e9c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f5c1015791423bbf99231d8cc6f903" is_pre_election: true
I20260808 10:45:00.935525 11247 raft_consensus.cc:2454] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7910b12be2c048c1bde2a069315e9c32 in term 0.
I20260808 10:45:00.935636 11660 raft_consensus.cc:2454] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7910b12be2c048c1bde2a069315e9c32 in term 0.
I20260808 10:45:00.936123 11351 leader_election.cc:302] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 11f5c1015791423bbf99231d8cc6f903, 7910b12be2c048c1bde2a069315e9c32, 8cff3352097c4b25a3805d484ad98c80; no voters:
I20260808 10:45:00.936470 11794 raft_consensus.cc:2781] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:45:00.936555 11794 raft_consensus.cc:482] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:45:00.936587 11794 raft_consensus.cc:3037] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:00.937685 11794 raft_consensus.cc:504] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:00.938098 11794 leader_election.cc:288] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [CANDIDATE]: Term 1 election: Requested vote from peers 11f5c1015791423bbf99231d8cc6f903 (127.31.237.131:45425), 8cff3352097c4b25a3805d484ad98c80 (127.31.237.129:32989)
I20260808 10:45:00.938462 11660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ef1c057742e4127919d5fd712883169" candidate_uuid: "7910b12be2c048c1bde2a069315e9c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f5c1015791423bbf99231d8cc6f903"
I20260808 10:45:00.938548 11247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ef1c057742e4127919d5fd712883169" candidate_uuid: "7910b12be2c048c1bde2a069315e9c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cff3352097c4b25a3805d484ad98c80"
I20260808 10:45:00.938607 11660 raft_consensus.cc:3037] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:00.938738 11247 raft_consensus.cc:3037] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:00.939713 11660 raft_consensus.cc:2454] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7910b12be2c048c1bde2a069315e9c32 in term 1.
I20260808 10:45:00.939730 11247 raft_consensus.cc:2454] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7910b12be2c048c1bde2a069315e9c32 in term 1.
I20260808 10:45:00.940006 11351 leader_election.cc:302] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7910b12be2c048c1bde2a069315e9c32, 8cff3352097c4b25a3805d484ad98c80; no voters:
I20260808 10:45:00.940187 11794 raft_consensus.cc:2781] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:45:00.940452 11794 raft_consensus.cc:686] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [term 1 LEADER]: Becoming Leader. State: Replica: 7910b12be2c048c1bde2a069315e9c32, State: Running, Role: LEADER
I20260808 10:45:00.941042 11794 consensus_queue.cc:237] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
W20260808 10:45:00.943315 11318 tablet.cc:2369] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:45:00.944736 9736 catalog_manager.cc:5495] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7910b12be2c048c1bde2a069315e9c32 (127.31.237.130). New cstate: current_term: 1 leader_uuid: "7910b12be2c048c1bde2a069315e9c32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } health_report { overall_health: HEALTHY } } }
I20260808 10:45:00.964346 11783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.960757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43610 (local address 127.31.237.130:35331)
0808 10:45:00.960886 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.960895 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:00.960968 (+ 73us) server_negotiation.cc:407] Connection header received
0808 10:45:00.961097 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:00.961100 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:00.961154 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:00.961243 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:45:00.961454 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.961945 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.962797 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:00.963096 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:00.963177 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:00.963454 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:45:00.963918 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:45:00.964043 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:00.964101 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 10:45:00.964160 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:45:00.970849 11247 raft_consensus.cc:1261] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80 [term 1 FOLLOWER]: Refusing update from remote peer 7910b12be2c048c1bde2a069315e9c32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:00.970852 11660 raft_consensus.cc:1261] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903 [term 1 FOLLOWER]: Refusing update from remote peer 7910b12be2c048c1bde2a069315e9c32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:00.971652 11794 consensus_queue.cc:1035] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:00.971877 11799 consensus_queue.cc:1035] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:00.980866 11808 mvcc.cc:205] Tried to move back new op lower bound from 7316217450368978944 to 7316217450257117184. Current Snapshot: MvccSnapshot[applied={T|T < 7316217450368978944}]
I20260808 10:45:00.985589 11806 mvcc.cc:205] Tried to move back new op lower bound from 7316217450368978944 to 7316217450257117184. Current Snapshot: MvccSnapshot[applied={T|T < 7316217450368978944}]
I20260808 10:45:00.987263 11804 mvcc.cc:205] Tried to move back new op lower bound from 7316217450368978944 to 7316217450257117184. Current Snapshot: MvccSnapshot[applied={T|T < 7316217450368978944 or (T in {7316217450368978944})}]
I20260808 10:45:00.990808 9736 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:45:00.991021 9736 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:45:00.995302 9736 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f908020858d84ab8a68a0c90fbc3ea5a: Sending DeleteTablet for 3 replicas of tablet 3ef1c057742e4127919d5fd712883169
I20260808 10:45:00.996124 11631 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ef1c057742e4127919d5fd712883169 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:00 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:53708
I20260808 10:45:00.996703 11815 tablet_replica.cc:330] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903: stopping tablet replica
I20260808 10:45:00.996724 11218 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ef1c057742e4127919d5fd712883169 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:00 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:45838
I20260808 10:45:00.997087 11815 raft_consensus.cc:2229] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:45:00.997517 11815 raft_consensus.cc:2258] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:00.997900 11424 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ef1c057742e4127919d5fd712883169 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:00 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:43604
I20260808 10:45:00.998965 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 9692
I20260808 10:45:01.011210 11815 ts_tablet_manager.cc:1905] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:01.012432 11821 tablet_replica.cc:330] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32: stopping tablet replica
I20260808 10:45:01.012883 11821 raft_consensus.cc:2229] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:45:01.013250 11817 tablet_replica.cc:330] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80: stopping tablet replica
I20260808 10:45:01.013361 11821 raft_consensus.cc:2258] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:01.016016 11815 ts_tablet_manager.cc:1918] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:45:01.016148 11815 log.cc:1196] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/3ef1c057742e4127919d5fd712883169
I20260808 10:45:01.016517 11815 ts_tablet_manager.cc:1939] T 3ef1c057742e4127919d5fd712883169 P 11f5c1015791423bbf99231d8cc6f903: Deleting consensus metadata
I20260808 10:45:01.018078 11817 raft_consensus.cc:2229] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:45:01.018563 11817 raft_consensus.cc:2258] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:01.020370 11817 ts_tablet_manager.cc:1905] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 10:45:01.021411 11568 connection.cc:535] server connection from 127.0.0.1:53708 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 10:45:01.021828 10681 connection.cc:535] server connection from 127.0.0.1:59778 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:45:01.024019 11817 ts_tablet_manager.cc:1918] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:45:01.024210 11817 log.cc:1196] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-0/wal/wals/3ef1c057742e4127919d5fd712883169
I20260808 10:45:01.024261 11821 ts_tablet_manager.cc:1905] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:01.024659 11817 ts_tablet_manager.cc:1939] T 3ef1c057742e4127919d5fd712883169 P 8cff3352097c4b25a3805d484ad98c80: Deleting consensus metadata
W20260808 10:45:01.025880 11151 connection.cc:441] server connection from 127.0.0.1:45838 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45838 (request call id 1) could send its response
I20260808 10:45:01.027348 11821 ts_tablet_manager.cc:1918] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:45:01.027545 11821 log.cc:1196] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/3ef1c057742e4127919d5fd712883169
I20260808 10:45:01.027963 11821 ts_tablet_manager.cc:1939] T 3ef1c057742e4127919d5fd712883169 P 7910b12be2c048c1bde2a069315e9c32: Deleting consensus metadata
W20260808 10:45:01.029286 11352 connection.cc:441] server connection from 127.0.0.1:43604 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43604 (request call id 1) could send its response
I20260808 10:45:01.037384 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:01.027255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47870 (local address 127.31.237.188:42141)
0808 10:45:01.027401 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:01.027406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:01.028161 (+ 755us) server_negotiation.cc:407] Connection header received
0808 10:45:01.028284 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:01.028287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:01.028341 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:01.028432 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:01.029653 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:01.030396 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:01.031472 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:01.031875 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:01.032002 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:01.033606 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:01.033610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:01.033613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:01.033640 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:01.035456 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:01.035959 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:01.035963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:01.035965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:01.036050 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:01.036365 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:01.036369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:01.036371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:01.036577 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:01.036676 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:01.037034 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 10:45:01.037199 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":51}
I20260808 10:45:01.043418 11749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.998252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59798 (local address 127.31.237.189:35269)
0808 10:45:00.998378 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.998382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:01.022687 (+ 24305us) server_negotiation.cc:407] Connection header received
0808 10:45:01.022855 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:01.022859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:01.022926 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:01.023019 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:01.024023 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:01.024899 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:01.026047 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:01.026422 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:01.026499 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:01.032851 (+ 6352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:01.032858 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:01.032864 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:01.032899 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:01.034546 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:01.035144 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:01.035148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:01.035150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:01.035283 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:01.035686 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:01.035691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:01.035692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:01.035941 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:01.043004 (+ 7063us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:01.043093 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 10:45:01.043239 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:45:01.045202 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:00.998390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47856 (local address 127.31.237.188:42141)
0808 10:45:00.998512 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:00.998516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:01.036785 (+ 38269us) server_negotiation.cc:407] Connection header received
0808 10:45:01.036965 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:01.036969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:01.037013 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:01.037061 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:01.037806 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:01.038559 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:01.039701 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:01.040025 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:01.040093 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:01.041695 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:01.041699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:01.041701 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:01.041725 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:01.043356 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:01.043897 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:01.043901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:01.043902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:01.043983 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:01.044287 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:01.044290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:01.044292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:01.044494 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:01.044606 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:01.044926 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 10:45:01.045034 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":38}
I20260808 10:45:01.045542 10919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:01.025511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59802 (local address 127.31.237.189:35269)
0808 10:45:01.025638 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:01.025642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:01.026331 (+ 689us) server_negotiation.cc:407] Connection header received
0808 10:45:01.026497 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:01.026500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:01.026550 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:01.026703 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:01.027885 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:01.028598 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:01.029538 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:01.029864 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:01.029970 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:01.032542 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:01.032546 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:01.032548 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:01.032577 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:01.036225 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:01.044199 (+ 7974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:01.044206 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:01.044211 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:01.044302 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:01.044625 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:01.044629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:01.044631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:01.044836 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:01.044933 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:01.045267 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 10:45:01.045396 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:45:01.985054 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:01.984509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.131:35449)
0808 10:45:01.984816 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:01.984924 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":118,"threads_started":1}
W20260808 10:45:01.985384 11729 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:36401 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
I20260808 10:45:01.988818 11834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:01.988296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.130:42123)
0808 10:45:01.988605 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:01.988700 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":101,"threads_started":1}
I20260808 10:45:01.989324 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:01.988848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.129:43385)
0808 10:45:01.989138 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:01.989227 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":62,"threads_started":1}
W20260808 10:45:01.989315 11528 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:36401 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
W20260808 10:45:01.989603 11315 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:36401 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
I20260808 10:45:02.495844 11839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f908020858d84ab8a68a0c90fbc3ea5a)
I20260808 10:45:02.496102 11839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } }
I20260808 10:45:02.497141 11839 leader_election.cc:288] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f908020858d84ab8a68a0c90fbc3ea5a (127.31.237.190:36401), e2ee44ced7704478a10c8c32f9eb9570 (127.31.237.189:35269)
I20260808 10:45:02.498042 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.497394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.0.0.1:60358)
0808 10:45:02.497757 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.497906 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":92,"threads_started":1}
W20260808 10:45:02.498304 11007 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
I20260808 10:45:02.499191 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.498942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.0.0.1:60374)
0808 10:45:02.499083 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.499132 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260808 10:45:02.499482 11007 leader_election.cc:334] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f908020858d84ab8a68a0c90fbc3ea5a (127.31.237.190:36401): Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
I20260808 10:45:02.499502 11846 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f908020858d84ab8a68a0c90fbc3ea5a)
I20260808 10:45:02.499702 11846 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } }
I20260808 10:45:02.500624 11846 leader_election.cc:288] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f908020858d84ab8a68a0c90fbc3ea5a (127.31.237.190:36401), 56098355de674fc0ae7dbdf27c5dc2e3 (127.31.237.188:42141)
I20260808 10:45:02.501116 11847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.500704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.0.0.1:60376)
0808 10:45:02.500912 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.500984 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":67,"threads_started":1}
W20260808 10:45:02.501402 10676 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
I20260808 10:45:02.502095 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.497536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.31.237.189:35269)
0808 10:45:02.497833 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:02.497838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:02.497861 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:45:02.498038 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:02.498042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:02.498111 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:02.498699 (+ 588us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:02.499027 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.499766 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.500859 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.501516 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.501653 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:02.501717 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:02.501893 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 10:45:02.501957 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":68,"threads_started":1}
I20260808 10:45:02.502106 11843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.497394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:35269 (local address 127.0.0.1:45028)
0808 10:45:02.497792 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.497806 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:02.497914 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:02.498720 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:02.498723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:02.498747 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:02.498898 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:02.498905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.499891 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.499897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:02.500734 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:02.500748 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.501626 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.501629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:02.501755 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:02.501786 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 10:45:02.501857 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:45:02.501934 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":44,"thread_start_us":57,"threads_started":1}
I20260808 10:45:02.502784 11849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.502540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.0.0.1:60386)
0808 10:45:02.502668 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.502724 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 10:45:02.502948 10676 leader_election.cc:334] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f908020858d84ab8a68a0c90fbc3ea5a (127.31.237.190:36401): Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
I20260808 10:45:02.502962 10721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e2ee44ced7704478a10c8c32f9eb9570" is_pre_election: true
I20260808 10:45:02.503217 10721 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56098355de674fc0ae7dbdf27c5dc2e3 in term 1.
I20260808 10:45:02.503604 11012 leader_election.cc:302] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56098355de674fc0ae7dbdf27c5dc2e3, e2ee44ced7704478a10c8c32f9eb9570; no voters: f908020858d84ab8a68a0c90fbc3ea5a
I20260808 10:45:02.503847 11839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 10:45:02.503952 11839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f908020858d84ab8a68a0c90fbc3ea5a)
I20260808 10:45:02.504012 11839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:45:02.505276 11847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.501059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42141 (local address 127.0.0.1:57428)
0808 10:45:02.501384 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.501400 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:02.501491 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:02.501804 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:02.501808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:02.501829 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:02.501919 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:02.501925 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.503028 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.503032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:02.504178 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:02.504189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.504999 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.505002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:02.505067 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:02.505087 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 10:45:02.505142 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 10:45:02.505189 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":53,"threads_started":1}
I20260808 10:45:02.505450 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.501224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57428 (local address 127.31.237.188:42141)
0808 10:45:02.501498 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:02.501507 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:02.501521 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:45:02.501568 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:02.501574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:02.501621 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:02.501695 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:02.502041 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.502845 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.504302 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.504869 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.504994 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:02.505040 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:02.505181 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 10:45:02.505281 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":115,"threads_started":1}
I20260808 10:45:02.505409 11839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } }
I20260808 10:45:02.505849 11839 leader_election.cc:288] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [CANDIDATE]: Term 2 election: Requested vote from peers f908020858d84ab8a68a0c90fbc3ea5a (127.31.237.190:36401), e2ee44ced7704478a10c8c32f9eb9570 (127.31.237.189:35269)
I20260808 10:45:02.505944 11054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e2ee44ced7704478a10c8c32f9eb9570" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" is_pre_election: true
I20260808 10:45:02.506184 11054 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2ee44ced7704478a10c8c32f9eb9570 in current term 2: Already voted for candidate 56098355de674fc0ae7dbdf27c5dc2e3 in this term.
I20260808 10:45:02.506552 10676 leader_election.cc:302] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2ee44ced7704478a10c8c32f9eb9570; no voters: 56098355de674fc0ae7dbdf27c5dc2e3, f908020858d84ab8a68a0c90fbc3ea5a
I20260808 10:45:02.506744 11843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.506493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.0.0.1:60396)
0808 10:45:02.506627 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.506679 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 10:45:02.506695 10721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e2ee44ced7704478a10c8c32f9eb9570"
I20260808 10:45:02.506799 11846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:45:02.507609 11843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.507401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.0.0.1:60404)
0808 10:45:02.507508 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.507554 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 10:45:02.507712 11007 leader_election.cc:334] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f908020858d84ab8a68a0c90fbc3ea5a (127.31.237.190:36401): Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
I20260808 10:45:02.508003 11846 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:45:02.508958 10721 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56098355de674fc0ae7dbdf27c5dc2e3 in term 2.
I20260808 10:45:02.509617 11012 leader_election.cc:302] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56098355de674fc0ae7dbdf27c5dc2e3, e2ee44ced7704478a10c8c32f9eb9570; no voters: f908020858d84ab8a68a0c90fbc3ea5a
I20260808 10:45:02.509820 11839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:45:02.510068 11839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [term 2 LEADER]: Becoming Leader. State: Replica: 56098355de674fc0ae7dbdf27c5dc2e3, State: Running, Role: LEADER
I20260808 10:45:02.510339 11839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } }
I20260808 10:45:02.511801 11852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56098355de674fc0ae7dbdf27c5dc2e3. Latest consensus state: current_term: 2 leader_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } } }
I20260808 10:45:02.512007 11852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:02.512338 11855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:45:02.515022 11855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:45:02.515377 11855 catalog_manager.cc:1259] Loaded cluster ID: 282a68a2c33345ef84eece0d29bd333f
I20260808 10:45:02.515447 11855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:45:02.515990 11855 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:45:02.516382 11855 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: Loaded TSK: 0
I20260808 10:45:02.517050 11855 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:45:02.572995 11033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:45:02.576381 11033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:45:02.579783 11843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.579481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.0.0.1:60430)
0808 10:45:02.579626 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.579697 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 10:45:02.579896 10721 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [term 2 FOLLOWER]: Refusing update from remote peer 56098355de674fc0ae7dbdf27c5dc2e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 10:45:02.580521 11839 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 10:45:02.580858 11843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.580555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.0.0.1:60442)
0808 10:45:02.580717 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.580781 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260808 10:45:02.580992 11007 consensus_peers.cc:487] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 -> Peer f908020858d84ab8a68a0c90fbc3ea5a (127.31.237.190:36401): Couldn't send request to peer f908020858d84ab8a68a0c90fbc3ea5a. Status: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 10:45:02.582190 11846 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56098355de674fc0ae7dbdf27c5dc2e3. Latest consensus state: current_term: 2 leader_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } } }
I20260808 10:45:02.582311 11846 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:45:02.583443 11839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } } }
I20260808 10:45:02.583572 11839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3 [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:02.584064 11846 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f908020858d84ab8a68a0c90fbc3ea5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36401 } } peers { permanent_uuid: "e2ee44ced7704478a10c8c32f9eb9570" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "56098355de674fc0ae7dbdf27c5dc2e3" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42141 } attrs { promote: false } } }
I20260808 10:45:02.584189 11846 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2ee44ced7704478a10c8c32f9eb9570 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:45:02.594291 11843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.589426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:45425 (local address 127.0.0.1:34462)
0808 10:45:02.589955 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.589970 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:02.590053 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:02.590514 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:02.590520 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:02.590542 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:02.590680 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:02.590690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.591764 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.591767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:02.592936 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:02.592946 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.593807 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.593811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:02.593959 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:02.593987 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 10:45:02.594071 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:45:02.594145 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":70}
I20260808 10:45:02.594352 11866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.589709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34462 (local address 127.31.237.131:45425)
0808 10:45:02.590240 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:02.590245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:02.590260 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:45:02.590320 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:02.590323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:02.590373 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:02.590796 (+ 423us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:02.590935 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.591647 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.593062 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.593668 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.593871 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:02.593925 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:02.594106 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 10:45:02.594204 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":108,"threads_started":1}
I20260808 10:45:02.594751 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.589701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:32989 (local address 127.0.0.1:58928)
0808 10:45:02.590200 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.590214 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:02.590283 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:02.590579 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:02.590582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:02.590601 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:02.590766 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:02.590775 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.592184 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.592188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:02.593345 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:02.593355 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.594252 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.594256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:02.594397 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:02.594425 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 10:45:02.594495 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 10:45:02.594583 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":26}
I20260808 10:45:02.594750 11867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.589970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58928 (local address 127.31.237.129:32989)
0808 10:45:02.590324 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:02.590330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:02.590346 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:45:02.590400 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:02.590402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:02.590450 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:02.590988 (+ 538us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:02.591116 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.592054 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.593496 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.594117 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.594399 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:02.594462 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:02.594523 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 10:45:02.594596 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":98,"threads_started":1}
I20260808 10:45:02.595232 11631 tablet_service.cc:1467] Processing CreateTablet for tablet 41c0dafafe3041e59ab62e0a27e72301 (DEFAULT_TABLE table=test-table [id=903ec448b7d144c38b35bc9be83ff246]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:02.595481 11218 tablet_service.cc:1467] Processing CreateTablet for tablet 41c0dafafe3041e59ab62e0a27e72301 (DEFAULT_TABLE table=test-table [id=903ec448b7d144c38b35bc9be83ff246]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:02.595556 11631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41c0dafafe3041e59ab62e0a27e72301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:02.595747 11218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41c0dafafe3041e59ab62e0a27e72301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:02.597046 11869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.590012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:35331 (local address 127.0.0.1:54448)
0808 10:45:02.591410 (+ 1398us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.591424 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:02.591519 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:02.592675 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:02.592678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:02.592696 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:02.592799 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:02.592804 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.593586 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.593591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:02.595948 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:02.595958 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.596634 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.596638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:02.596736 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:02.596758 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:45:02.596828 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 10:45:02.596908 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1336,"mutex_wait_us":4,"thread_start_us":105,"threads_started":1}
I20260808 10:45:02.597095 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.590369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54448 (local address 127.31.237.130:35331)
0808 10:45:02.590884 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:02.590889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:02.591512 (+ 623us) server_negotiation.cc:407] Connection header received
0808 10:45:02.591601 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:02.591603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:02.591656 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:02.591744 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:02.592929 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.593473 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.596073 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.596507 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.596655 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:02.596714 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:02.596865 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 10:45:02.596949 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":76,"threads_started":1}
I20260808 10:45:02.598121 11424 tablet_service.cc:1467] Processing CreateTablet for tablet 41c0dafafe3041e59ab62e0a27e72301 (DEFAULT_TABLE table=test-table [id=903ec448b7d144c38b35bc9be83ff246]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:02.598443 11424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41c0dafafe3041e59ab62e0a27e72301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:02.600669 11875 tablet_bootstrap.cc:492] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80: Bootstrap starting.
I20260808 10:45:02.601219 11877 tablet_bootstrap.cc:492] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32: Bootstrap starting.
I20260808 10:45:02.601500 11873 tablet_bootstrap.cc:492] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903: Bootstrap starting.
I20260808 10:45:02.601665 11875 tablet_bootstrap.cc:654] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:02.602317 11877 tablet_bootstrap.cc:654] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:02.602566 11873 tablet_bootstrap.cc:654] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:02.603596 11875 tablet_bootstrap.cc:492] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80: No bootstrap required, opened a new log
I20260808 10:45:02.603725 11877 tablet_bootstrap.cc:492] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32: No bootstrap required, opened a new log
I20260808 10:45:02.603744 11875 ts_tablet_manager.cc:1397] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:45:02.603818 11877 ts_tablet_manager.cc:1397] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:45:02.604255 11875 raft_consensus.cc:348] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:02.604269 11877 raft_consensus.cc:348] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:02.604382 11875 raft_consensus.cc:374] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:02.604382 11877 raft_consensus.cc:374] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:02.604418 11877 raft_consensus.cc:729] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7910b12be2c048c1bde2a069315e9c32, State: Initialized, Role: FOLLOWER
I20260808 10:45:02.604418 11875 raft_consensus.cc:729] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cff3352097c4b25a3805d484ad98c80, State: Initialized, Role: FOLLOWER
I20260808 10:45:02.604566 11877 consensus_queue.cc:260] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:02.604566 11875 consensus_queue.cc:260] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:02.604931 11875 ts_tablet_manager.cc:1428] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:45:02.604930 11877 ts_tablet_manager.cc:1428] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:45:02.607031 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.605443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.129:44417)
0808 10:45:02.606879 (+ 1436us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.606937 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1356,"thread_start_us":97,"threads_started":1}
I20260808 10:45:02.607487 11885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.605210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.130:48243)
0808 10:45:02.607304 (+ 2094us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.607364 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2010,"thread_start_us":106,"threads_started":1}
I20260808 10:45:02.608283 11885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.607966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.130:57177)
0808 10:45:02.608152 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.608210 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 10:45:02.608413 11530 heartbeater.cc:503] Master 127.31.237.188:42141 was elected leader, sending a full tablet report...
I20260808 10:45:02.608654 11873 tablet_bootstrap.cc:492] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903: No bootstrap required, opened a new log
I20260808 10:45:02.608758 11873 ts_tablet_manager.cc:1397] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260808 10:45:02.609261 11873 raft_consensus.cc:348] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:02.609395 11873 raft_consensus.cc:374] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:02.609442 11873 raft_consensus.cc:729] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f5c1015791423bbf99231d8cc6f903, State: Initialized, Role: FOLLOWER
I20260808 10:45:02.609582 11873 consensus_queue.cc:260] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:02.609954 11873 ts_tablet_manager.cc:1428] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:45:02.610348 11316 heartbeater.cc:503] Master 127.31.237.188:42141 was elected leader, sending a full tablet report...
I20260808 10:45:02.612049 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.611736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.129:40879)
0808 10:45:02.611867 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.611972 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 10:45:02.612305 11730 heartbeater.cc:503] Master 127.31.237.188:42141 was elected leader, sending a full tablet report...
I20260808 10:45:02.613411 11890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.610293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.131:36025)
0808 10:45:02.613207 (+ 2914us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.613297 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2825,"thread_start_us":94,"threads_started":1}
I20260808 10:45:02.614140 11890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.613949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.131:39073)
0808 10:45:02.614040 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.614087 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 10:45:02.656577 11882 raft_consensus.cc:482] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:45:02.656730 11882 raft_consensus.cc:504] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:02.657266 11882 leader_election.cc:288] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11f5c1015791423bbf99231d8cc6f903 (127.31.237.131:45425), 8cff3352097c4b25a3805d484ad98c80 (127.31.237.129:32989)
I20260808 10:45:02.657826 11247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41c0dafafe3041e59ab62e0a27e72301" candidate_uuid: "7910b12be2c048c1bde2a069315e9c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cff3352097c4b25a3805d484ad98c80" is_pre_election: true
I20260808 10:45:02.657855 11660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41c0dafafe3041e59ab62e0a27e72301" candidate_uuid: "7910b12be2c048c1bde2a069315e9c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f5c1015791423bbf99231d8cc6f903" is_pre_election: true
I20260808 10:45:02.658023 11247 raft_consensus.cc:2454] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7910b12be2c048c1bde2a069315e9c32 in term 0.
I20260808 10:45:02.658036 11660 raft_consensus.cc:2454] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7910b12be2c048c1bde2a069315e9c32 in term 0.
I20260808 10:45:02.658392 11351 leader_election.cc:302] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7910b12be2c048c1bde2a069315e9c32, 8cff3352097c4b25a3805d484ad98c80; no voters:
I20260808 10:45:02.658667 11882 raft_consensus.cc:2781] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:45:02.658737 11882 raft_consensus.cc:482] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:45:02.658764 11882 raft_consensus.cc:3037] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:02.659968 11882 raft_consensus.cc:504] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:02.660353 11882 leader_election.cc:288] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [CANDIDATE]: Term 1 election: Requested vote from peers 11f5c1015791423bbf99231d8cc6f903 (127.31.237.131:45425), 8cff3352097c4b25a3805d484ad98c80 (127.31.237.129:32989)
I20260808 10:45:02.660771 11247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41c0dafafe3041e59ab62e0a27e72301" candidate_uuid: "7910b12be2c048c1bde2a069315e9c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cff3352097c4b25a3805d484ad98c80"
I20260808 10:45:02.660910 11247 raft_consensus.cc:3037] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:02.660997 11660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41c0dafafe3041e59ab62e0a27e72301" candidate_uuid: "7910b12be2c048c1bde2a069315e9c32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f5c1015791423bbf99231d8cc6f903"
I20260808 10:45:02.661115 11660 raft_consensus.cc:3037] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:02.661827 11247 raft_consensus.cc:2454] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7910b12be2c048c1bde2a069315e9c32 in term 1.
I20260808 10:45:02.662127 11660 raft_consensus.cc:2454] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7910b12be2c048c1bde2a069315e9c32 in term 1.
I20260808 10:45:02.662168 11351 leader_election.cc:302] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7910b12be2c048c1bde2a069315e9c32, 8cff3352097c4b25a3805d484ad98c80; no voters:
I20260808 10:45:02.662365 11882 raft_consensus.cc:2781] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:45:02.662695 11882 raft_consensus.cc:686] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [term 1 LEADER]: Becoming Leader. State: Replica: 7910b12be2c048c1bde2a069315e9c32, State: Running, Role: LEADER
I20260808 10:45:02.662880 11882 consensus_queue.cc:237] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } }
I20260808 10:45:02.663514 11885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.663216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.130:46133)
0808 10:45:02.663381 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.663447 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 10:45:02.664744 11034 catalog_manager.cc:5495] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7910b12be2c048c1bde2a069315e9c32 (127.31.237.130). New cstate: current_term: 1 leader_uuid: "7910b12be2c048c1bde2a069315e9c32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7910b12be2c048c1bde2a069315e9c32" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 35331 } health_report { overall_health: HEALTHY } } }
I20260808 10:45:02.681746 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.678546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54464 (local address 127.31.237.130:35331)
0808 10:45:02.678705 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:02.678710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:02.678794 (+ 84us) server_negotiation.cc:407] Connection header received
0808 10:45:02.678852 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:02.678855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:02.678899 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:02.678976 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:45:02.679244 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.679761 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.680634 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:02.680866 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:02.680982 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:02.681241 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:45:02.681313 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:45:02.681391 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:02.681528 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 10:45:02.681583 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:45:02.684073 11247 raft_consensus.cc:1261] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80 [term 1 FOLLOWER]: Refusing update from remote peer 7910b12be2c048c1bde2a069315e9c32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:02.684073 11660 raft_consensus.cc:1261] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903 [term 1 FOLLOWER]: Refusing update from remote peer 7910b12be2c048c1bde2a069315e9c32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:02.684561 11891 consensus_queue.cc:1035] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f5c1015791423bbf99231d8cc6f903" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:02.684783 11882 consensus_queue.cc:1035] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cff3352097c4b25a3805d484ad98c80" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 32989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:02.686740 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.686404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.129:49243)
0808 10:45:02.686565 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.686662 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 10:45:02.687359 11890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.687013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.131:51269)
0808 10:45:02.687233 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.687285 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260808 10:45:02.690843 11890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.690140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.131:54049)
0808 10:45:02.690716 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.690772 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496}
I20260808 10:45:02.691151 11885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.690293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.130:44987)
0808 10:45:02.691029 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.691081 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":662}
I20260808 10:45:02.692031 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.691750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:36401 (local address 127.31.237.129:57261)
0808 10:45:02.691896 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.691959 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:36401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260808 10:45:02.696810 11032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:45:02.696949 11032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:45:02.700568 11032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56098355de674fc0ae7dbdf27c5dc2e3: Sending DeleteTablet for 3 replicas of tablet 41c0dafafe3041e59ab62e0a27e72301
I20260808 10:45:02.701400 11218 tablet_service.cc:1514] Processing DeleteTablet for tablet 41c0dafafe3041e59ab62e0a27e72301 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:02 UTC) from {username='oryx', principal='oryx/127.31.237.188@KRBTEST.COM'} at 127.0.0.1:58928
I20260808 10:45:02.701467 11631 tablet_service.cc:1514] Processing DeleteTablet for tablet 41c0dafafe3041e59ab62e0a27e72301 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:02 UTC) from {username='oryx', principal='oryx/127.31.237.188@KRBTEST.COM'} at 127.0.0.1:34462
I20260808 10:45:02.701778 11910 tablet_replica.cc:330] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80: stopping tablet replica
I20260808 10:45:02.701956 11910 raft_consensus.cc:2229] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:45:02.701988 11424 tablet_service.cc:1514] Processing DeleteTablet for tablet 41c0dafafe3041e59ab62e0a27e72301 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:02 UTC) from {username='oryx', principal='oryx/127.31.237.188@KRBTEST.COM'} at 127.0.0.1:54448
I20260808 10:45:02.702131 11910 raft_consensus.cc:2258] T 41c0dafafe3041e59ab62e0a27e72301 P 8cff3352097c4b25a3805d484ad98c80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:02.702361 11912 tablet_replica.cc:330] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32: stopping tablet replica
I20260808 10:45:02.702500 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 10963
I20260808 10:45:02.702507 11912 raft_consensus.cc:2229] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:45:02.715572 11912 raft_consensus.cc:2258] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:02.716250 11911 tablet_replica.cc:330] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903: stopping tablet replica
I20260808 10:45:02.716447 11911 raft_consensus.cc:2229] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:45:02.716535 11912 ts_tablet_manager.cc:1905] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:02.716609 11911 raft_consensus.cc:2258] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:02.717458 11911 ts_tablet_manager.cc:1905] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:02.719192 11912 ts_tablet_manager.cc:1918] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:45:02.719301 11912 log.cc:1196] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/41c0dafafe3041e59ab62e0a27e72301
I20260808 10:45:02.719671 11912 ts_tablet_manager.cc:1939] T 41c0dafafe3041e59ab62e0a27e72301 P 7910b12be2c048c1bde2a069315e9c32: Deleting consensus metadata
I20260808 10:45:02.720501 11007 catalog_manager.cc:4841] TS 7910b12be2c048c1bde2a069315e9c32 (127.31.237.130:35331): tablet 41c0dafafe3041e59ab62e0a27e72301 (table test-table [id=903ec448b7d144c38b35bc9be83ff246]) successfully deleted
I20260808 10:45:02.720853 11911 ts_tablet_manager.cc:1918] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:45:02.720947 11911 log.cc:1196] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/41c0dafafe3041e59ab62e0a27e72301
I20260808 10:45:02.721235 11911 ts_tablet_manager.cc:1939] T 41c0dafafe3041e59ab62e0a27e72301 P 11f5c1015791423bbf99231d8cc6f903: Deleting consensus metadata
I20260808 10:45:02.722028 11007 catalog_manager.cc:4841] TS 11f5c1015791423bbf99231d8cc6f903 (127.31.237.131:45425): tablet 41c0dafafe3041e59ab62e0a27e72301 (table test-table [id=903ec448b7d144c38b35bc9be83ff246]) successfully deleted
I20260808 10:45:02.723567 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 11327
I20260808 10:45:02.723765 11869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:02.723473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:32989 (local address 127.0.0.1:58940)
0808 10:45:02.723589 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:02.723665 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:32989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 10:45:02.728982 11010 catalog_manager.cc:4584] TS 8cff3352097c4b25a3805d484ad98c80 (127.31.237.129:32989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 41c0dafafe3041e59ab62e0a27e72301: Network error: Client connection negotiation failed: client connection to 127.31.237.129:32989: connect: Connection refused (error 111)
I20260808 10:45:02.736572 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 11541
I20260808 10:45:02.749974 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 10489
I20260808 10:45:02.763338 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 10657
2026-08-08T10:45:02Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5836 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:45:02 dist-test-slave-twwt krb5kdc[11916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:45:02 dist-test-slave-twwt krb5kdc[11916](info): set up 2 sockets
Aug 08 10:45:02 dist-test-slave-twwt krb5kdc[11916](info): commencing operation
krb5kdc: starting...
W20260808 10:45:04.018913 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:45:04 dist-test-slave-twwt krb5kdc[11916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:45:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:45:04Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.190.keytab.
Entry for principal oryx/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.190.keytab.
I20260808 10:45:04.157491 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:45503
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.190.keytab
--principal=oryx/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:39081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:45503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:45:04.273519 11942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:04.273833 11942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:04.273912 11942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:04.278108 11942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:45:04.278208 11942 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:45:04.278237 11942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:04.278282 11942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:04.278316 11942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:45:04.278370 11942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:45:04.284050 11942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:39081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:45503
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.190.keytab
--principal=oryx/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:45503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:04.285375 11942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:04.286453 11942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:04.295248 11951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:04.295282 11958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:04.295460 11942 server_base.cc:1034] running on GCE node
W20260808 10:45:04.295334 11950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:04.296197 11942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:04.297371 11942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:04.298585 11942 hybrid_clock.cc:648] HybridClock initialized: now 1786185904298565 us; error 30 us; skew 500 ppm
Aug 08 10:45:04 dist-test-slave-twwt krb5kdc[11916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185904, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:04.301551 11942 init.cc:377] Logged in from keytab as oryx/127.31.237.190@KRBTEST.COM (short username oryx)
I20260808 10:45:04.302598 11942 webserver.cc:460] Webserver started at http://127.31.237.190:35501/ using document root <none> and password file <none>
I20260808 10:45:04.303321 11942 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:04.303422 11942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:04.303689 11942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:04.306030 11942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "431640e365114e1aaf692a07609e9ca0"
format_stamp: "Formatted at 2026-08-08 10:45:04 on dist-test-slave-twwt"
server_key: "eeb75007ad500409f4189d5debd3e54f"
server_key_iv: "069567a6c55b5b417496a1bb62458132"
server_key_version: "encryptionkey@0"
I20260808 10:45:04.306740 11942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "431640e365114e1aaf692a07609e9ca0"
format_stamp: "Formatted at 2026-08-08 10:45:04 on dist-test-slave-twwt"
server_key: "eeb75007ad500409f4189d5debd3e54f"
server_key_iv: "069567a6c55b5b417496a1bb62458132"
server_key_version: "encryptionkey@0"
I20260808 10:45:04.310956 11942 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:45:04.313582 11970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:04.315084 11942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:45:04.315261 11942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "431640e365114e1aaf692a07609e9ca0"
format_stamp: "Formatted at 2026-08-08 10:45:04 on dist-test-slave-twwt"
server_key: "eeb75007ad500409f4189d5debd3e54f"
server_key_iv: "069567a6c55b5b417496a1bb62458132"
server_key_version: "encryptionkey@0"
I20260808 10:45:04.315392 11942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:04.344622 11942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:04.348479 11942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:04.348721 11942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:04.357241 11942 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:45503
I20260808 10:45:04.357263 12068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:45503 every 8 connection(s)
I20260808 10:45:04.358242 11942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:45:04.361502 12069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:04.365543 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 11942
I20260808 10:45:04.365705 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:45:04.366003 32694 external_mini_cluster.cc:1428] Setting key c49d7a2d877a2e23de32b777c1f9cf65
I20260808 10:45:04.368147 12069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0: Bootstrap starting.
I20260808 10:45:04.371084 12069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:04.371989 12069 log.cc:824] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:04.374656 12069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0: No bootstrap required, opened a new log
Aug 08 10:45:04 dist-test-slave-twwt krb5kdc[11916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
I20260808 10:45:04.378808 12069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431640e365114e1aaf692a07609e9ca0" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45503 } }
I20260808 10:45:04.379066 12069 raft_consensus.cc:374] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:04.379159 12069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431640e365114e1aaf692a07609e9ca0, State: Initialized, Role: FOLLOWER
I20260808 10:45:04.379705 12069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431640e365114e1aaf692a07609e9ca0" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45503 } }
I20260808 10:45:04.379876 12069 raft_consensus.cc:388] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:45:04.379959 12069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:45:04.380077 12069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:04.381469 12069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431640e365114e1aaf692a07609e9ca0" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45503 } }
I20260808 10:45:04.381882 12069 leader_election.cc:302] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 431640e365114e1aaf692a07609e9ca0; no voters:
I20260808 10:45:04.382292 12069 leader_election.cc:288] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:45:04.382465 12078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:45:04.382663 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:04.367600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39174 (local address 127.31.237.190:45503)
0808 10:45:04.368049 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:04.368065 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:04.368104 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:45:04.368796 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:04.368817 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:04.369171 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:04.369484 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:04.370532 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.371756 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.372939 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.373475 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.373602 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:04.376884 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:04.376890 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:04.376905 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:04.376935 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:04.378901 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:04.379433 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:04.379440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:04.379447 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:04.379564 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:04.379909 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:04.379913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:04.379916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:04.380539 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:04.380794 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:04.381069 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 10:45:04.381444 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":142,"threads_started":1}
I20260808 10:45:04.382823 12078 raft_consensus.cc:686] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [term 1 LEADER]: Becoming Leader. State: Replica: 431640e365114e1aaf692a07609e9ca0, State: Running, Role: LEADER
I20260808 10:45:04.383215 12078 consensus_queue.cc:237] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431640e365114e1aaf692a07609e9ca0" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45503 } }
I20260808 10:45:04.383805 12069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:45:04.385815 12082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 431640e365114e1aaf692a07609e9ca0. Latest consensus state: current_term: 1 leader_uuid: "431640e365114e1aaf692a07609e9ca0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431640e365114e1aaf692a07609e9ca0" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45503 } } }
I20260808 10:45:04.385986 12082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:04.386294 12081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "431640e365114e1aaf692a07609e9ca0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "431640e365114e1aaf692a07609e9ca0" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45503 } } }
I20260808 10:45:04.386435 12097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:45:04.386479 12081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0 [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:04.389430 12097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:45:04.395767 12097 catalog_manager.cc:1347] Generated new cluster ID: cd7e0f421fd44e759e8b89ed8d4d2f56
I20260808 10:45:04.395845 12097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:45:04.425894 12097 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:45:04.427306 12097 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:45:04.438318 12097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0: Generated new TSK 0
I20260808 10:45:04.439221 12097 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.129.keytab.
Entry for principal oryx/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.129.keytab.
I20260808 10:45:04.474850 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:45503
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.129.keytab
--principal=oryx/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:39081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:45:04.597504 12113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:04.597812 12113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:04.597859 12113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:04.601994 12113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:04.602059 12113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:04.602100 12113 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:45:04.602134 12113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:45:04.607483 12113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:39081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.129.keytab
--principal=oryx/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.237.190:45503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:04.608582 12113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:04.609629 12113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:04.619992 12124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:04.620002 12121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:04.619987 12120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:04.620327 12113 server_base.cc:1034] running on GCE node
I20260808 10:45:04.620926 12113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:04.621649 12113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:04.622859 12113 hybrid_clock.cc:648] HybridClock initialized: now 1786185904622838 us; error 36 us; skew 500 ppm
Aug 08 10:45:04 dist-test-slave-twwt krb5kdc[11916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185904, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:04.625984 12113 init.cc:377] Logged in from keytab as oryx/127.31.237.129@KRBTEST.COM (short username oryx)
I20260808 10:45:04.627079 12113 webserver.cc:460] Webserver started at http://127.31.237.129:33083/ using document root <none> and password file <none>
I20260808 10:45:04.627686 12113 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:04.627765 12113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:04.627954 12113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:04.630077 12113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1"
format_stamp: "Formatted at 2026-08-08 10:45:04 on dist-test-slave-twwt"
server_key: "5649779f58fccd1b151f0f54f0f3e0e5"
server_key_iv: "26563693560f85bc89b3721df1bb2a70"
server_key_version: "encryptionkey@0"
I20260808 10:45:04.630694 12113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1"
format_stamp: "Formatted at 2026-08-08 10:45:04 on dist-test-slave-twwt"
server_key: "5649779f58fccd1b151f0f54f0f3e0e5"
server_key_iv: "26563693560f85bc89b3721df1bb2a70"
server_key_version: "encryptionkey@0"
I20260808 10:45:04.634950 12113 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:45:04.637660 12131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:04.639055 12113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:45:04.639158 12113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1"
format_stamp: "Formatted at 2026-08-08 10:45:04 on dist-test-slave-twwt"
server_key: "5649779f58fccd1b151f0f54f0f3e0e5"
server_key_iv: "26563693560f85bc89b3721df1bb2a70"
server_key_version: "encryptionkey@0"
I20260808 10:45:04.639251 12113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:04.672379 12113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:04.676014 12113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:04.676180 12113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:04.677695 12113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:04.677757 12113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:04.677826 12113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:04.677860 12113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:45:04 dist-test-slave-twwt krb5kdc[11916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185904, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
I20260808 10:45:04.689465 12113 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:43229
I20260808 10:45:04.689528 12328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:43229 every 8 connection(s)
I20260808 10:45:04.690685 12113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:45:04.693121 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:04.678504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:60713 (local address 127.31.237.190:45503)
0808 10:45:04.678710 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:04.678718 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:04.679669 (+ 951us) server_negotiation.cc:407] Connection header received
0808 10:45:04.681011 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:04.681018 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:04.681092 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:04.681157 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:04.683209 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.683954 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.685946 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.686335 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.686432 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:04.689277 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:04.689281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:04.689286 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:04.689319 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:04.691014 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:04.691647 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:04.691651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:04.691653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:04.691752 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:04.692147 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:04.692149 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:04.692150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:04.692349 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:04.692426 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:04.692898 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 10:45:04.693003 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:45:04.693346 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 12113
I20260808 10:45:04.693495 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:45:04.693819 32694 external_mini_cluster.cc:1428] Setting key 7c635db572d6e7313f35257edad9cacf
I20260808 10:45:04.694197 12146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:04.678962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:45503 (local address 127.31.237.129:60713)
0808 10:45:04.679482 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:04.679543 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:04.680731 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:04.681644 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:04.681656 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:04.682138 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:04.683027 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:04.683054 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.684116 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.684120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:04.685808 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:04.685824 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.686505 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.686511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:04.686768 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:04.687484 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:04.687510 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:04.689061 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:04.691130 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:04.691136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:04.691150 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:04.691519 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:04.691873 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:04.691877 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:04.691879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:04.692051 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:04.692444 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:04.692450 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:04.692670 (+ 220us) client_negotiation.cc:769] Sending connection context
0808 10:45:04.692833 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 10:45:04.693126 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":106,"threads_started":1}
I20260808 10:45:04.695502 12329 heartbeater.cc:347] Connected to a master server at 127.31.237.190:45503
I20260808 10:45:04.695672 12329 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:04.696170 12329 heartbeater.cc:511] Master 127.31.237.190:45503 requested a full tablet report, sending...
I20260808 10:45:04.697759 11996 ts_manager.cc:194] Registered new tserver with Master: 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129:43229)
I20260808 10:45:04.699375 11996 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:60713
I20260808 10:45:04.706182 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:04.698069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:40391 (local address 127.31.237.190:45503)
0808 10:45:04.698189 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:04.698193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:04.698434 (+ 241us) server_negotiation.cc:407] Connection header received
0808 10:45:04.698583 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:04.698586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:04.698656 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:04.698726 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:04.699214 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.699778 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.701088 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.701411 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.701521 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:04.702935 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:04.702946 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:04.702948 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:04.702970 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:04.704483 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:04.704983 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:04.704986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:04.704987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:04.705063 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:04.705384 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:04.705386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:04.705387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:04.705552 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:04.705626 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:04.705969 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 10:45:04.706064 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for oryx/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:45:04.706292 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:04.698016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:45503 (local address 127.31.237.129:40391)
0808 10:45:04.698348 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:04.698359 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:04.698475 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:04.698836 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:04.698838 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:04.699007 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:04.699098 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:04.699103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.699904 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.699910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:04.700947 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:04.700962 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.701554 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.701561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:04.701703 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:04.702210 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:04.702226 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:04.702741 (+ 515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:04.704602 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:04.704607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:04.704610 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:04.704880 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:04.705164 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:04.705167 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:04.705169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:04.705278 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:04.705657 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:04.705663 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:04.705792 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 10:45:04.705915 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 10:45:04.706072 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.237.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.130.keytab.
Entry for principal oryx/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.130.keytab.
I20260808 10:45:04.725865 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:45503
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.130.keytab
--principal=oryx/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:39081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:45:04.845660 12343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:04.845955 12343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:04.846001 12343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:04.850165 12343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:04.850232 12343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:04.850274 12343 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:45:04.850349 12343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:45:04.855923 12343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:39081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.130.keytab
--principal=oryx/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.237.190:45503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:04.857086 12343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:04.858223 12343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:04.868459 12355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:04.868511 12350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:04.868465 12351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:04.868681 12343 server_base.cc:1034] running on GCE node
I20260808 10:45:04.869320 12343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:04.870023 12343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:04.871214 12343 hybrid_clock.cc:648] HybridClock initialized: now 1786185904871192 us; error 45 us; skew 500 ppm
Aug 08 10:45:04 dist-test-slave-twwt krb5kdc[11916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185904, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:04.873988 12343 init.cc:377] Logged in from keytab as oryx/127.31.237.130@KRBTEST.COM (short username oryx)
I20260808 10:45:04.874907 12343 webserver.cc:460] Webserver started at http://127.31.237.130:44295/ using document root <none> and password file <none>
I20260808 10:45:04.875456 12343 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:04.875515 12343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:04.875685 12343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:04.877677 12343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "6710af7e2a094b979525619766dfb2ae"
format_stamp: "Formatted at 2026-08-08 10:45:04 on dist-test-slave-twwt"
server_key: "443f9f1af018acd09acf6502fe3c7805"
server_key_iv: "5d962facd0a5ea44db609f86e2435ae7"
server_key_version: "encryptionkey@0"
I20260808 10:45:04.878198 12343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "6710af7e2a094b979525619766dfb2ae"
format_stamp: "Formatted at 2026-08-08 10:45:04 on dist-test-slave-twwt"
server_key: "443f9f1af018acd09acf6502fe3c7805"
server_key_iv: "5d962facd0a5ea44db609f86e2435ae7"
server_key_version: "encryptionkey@0"
I20260808 10:45:04.882045 12343 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:45:04.884778 12367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:04.886206 12343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:45:04.886340 12343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "6710af7e2a094b979525619766dfb2ae"
format_stamp: "Formatted at 2026-08-08 10:45:04 on dist-test-slave-twwt"
server_key: "443f9f1af018acd09acf6502fe3c7805"
server_key_iv: "5d962facd0a5ea44db609f86e2435ae7"
server_key_version: "encryptionkey@0"
I20260808 10:45:04.886493 12343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:04.899063 12343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:04.902390 12343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:04.902557 12343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:04.904098 12343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:04.904155 12343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:04.904223 12343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:04.904264 12343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:45:04 dist-test-slave-twwt krb5kdc[11916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185904, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
I20260808 10:45:04.915570 12343 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:37139
I20260808 10:45:04.915618 12557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:37139 every 8 connection(s)
I20260808 10:45:04.916466 12343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:45:04.919431 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:04.905124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:37075 (local address 127.31.237.190:45503)
0808 10:45:04.905262 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:04.905266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:04.906406 (+ 1140us) server_negotiation.cc:407] Connection header received
0808 10:45:04.907715 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:04.907719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:04.907795 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:04.907902 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:04.909499 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.910404 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.911838 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.912079 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.912199 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:04.915460 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:04.915464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:04.915467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:04.915501 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:04.917239 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:04.917860 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:04.917863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:04.917864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:04.917945 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:04.918249 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:04.918251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:04.918252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:04.918448 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:04.918528 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:04.919123 (+ 595us) server_negotiation.cc:299] Negotiation successful
0808 10:45:04.919232 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:45:04.920133 12384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:04.905713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:45503 (local address 127.31.237.130:37075)
0808 10:45:04.906234 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:04.906281 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:04.907375 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:04.908122 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:04.908134 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:04.908585 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:04.909328 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:04.909354 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.910563 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.910566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:04.911705 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:04.911715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.912252 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.912257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:04.912441 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:04.913121 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:04.913168 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:04.915223 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:04.917499 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:04.917503 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:04.917513 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:04.917758 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:04.918025 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:04.918027 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:04.918028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:04.918161 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:04.918540 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:04.918545 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:04.918854 (+ 309us) client_negotiation.cc:769] Sending connection context
0808 10:45:04.919003 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 10:45:04.919283 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":147,"threads_started":1}
I20260808 10:45:04.921478 12559 heartbeater.cc:347] Connected to a master server at 127.31.237.190:45503
I20260808 10:45:04.921635 12559 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:04.922152 12559 heartbeater.cc:511] Master 127.31.237.190:45503 requested a full tablet report, sending...
I20260808 10:45:04.923321 11996 ts_manager.cc:194] Registered new tserver with Master: 6710af7e2a094b979525619766dfb2ae (127.31.237.130:37139)
I20260808 10:45:04.923573 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 12343
I20260808 10:45:04.923710 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:45:04.924084 32694 external_mini_cluster.cc:1428] Setting key 6e15b530da3286fab0e54f28d416522f
I20260808 10:45:04.924075 11996 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:37075
I20260808 10:45:04.931953 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:04.924120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:42111 (local address 127.31.237.190:45503)
0808 10:45:04.924233 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:04.924238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:04.924252 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:45:04.924304 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:04.924307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:04.924352 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:04.924433 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:04.925272 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.925976 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.927002 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.927221 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.927353 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:04.928537 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:04.928544 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:04.928550 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:04.928581 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:04.930105 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:04.930682 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:04.930685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:04.930686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:04.930763 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:04.931076 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:04.931078 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:04.931079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:04.931276 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:04.931370 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:04.931725 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 10:45:04.931821 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:45:04.932117 12566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:04.923752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:45503 (local address 127.31.237.130:42111)
0808 10:45:04.924054 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:04.924069 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:04.924226 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:04.924732 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:04.924735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:04.924944 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:04.925122 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:04.925131 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.926144 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.926147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:04.926881 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:04.926891 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:04.927400 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:04.927406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:04.927529 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:04.928061 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:04.928073 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:04.928395 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:04.930222 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:04.930228 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:04.930233 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:04.930534 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:04.930864 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:04.930869 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:04.930872 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:04.930978 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:04.931387 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:04.931393 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:04.931522 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 10:45:04.931656 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 10:45:04.931834 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":13,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for oryx/127.31.237.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.237.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.131.keytab.
Entry for principal oryx/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.131.keytab.
I20260808 10:45:04.954808 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:45503
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.131.keytab
--principal=oryx/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:39081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:45:05.072659 12573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:05.072952 12573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:05.072999 12573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:05.078251 12573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:05.078342 12573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:05.078411 12573 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:45:05.078482 12573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:45:05.083912 12573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:39081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.131.keytab
--principal=oryx/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.237.190:45503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:05.085254 12573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:05.086377 12573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:05.096671 12580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:05.096675 12582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:05.096731 12579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:05.097080 12573 server_base.cc:1034] running on GCE node
I20260808 10:45:05.097489 12573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:05.098142 12573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:05.099323 12573 hybrid_clock.cc:648] HybridClock initialized: now 1786185905099288 us; error 49 us; skew 500 ppm
Aug 08 10:45:05 dist-test-slave-twwt krb5kdc[11916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185905, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:05.102229 12573 init.cc:377] Logged in from keytab as oryx/127.31.237.131@KRBTEST.COM (short username oryx)
I20260808 10:45:05.103143 12573 webserver.cc:460] Webserver started at http://127.31.237.131:33033/ using document root <none> and password file <none>
I20260808 10:45:05.103680 12573 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:05.103739 12573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:05.103901 12573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:05.105950 12573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "b2340a1811ce46fc857fd0ba432de650"
format_stamp: "Formatted at 2026-08-08 10:45:05 on dist-test-slave-twwt"
server_key: "1e909b9440f3b3376d2e7e6e82d5c31f"
server_key_iv: "66a03228e60ee6cdcfd23beb9539387d"
server_key_version: "encryptionkey@0"
I20260808 10:45:05.106498 12573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "b2340a1811ce46fc857fd0ba432de650"
format_stamp: "Formatted at 2026-08-08 10:45:05 on dist-test-slave-twwt"
server_key: "1e909b9440f3b3376d2e7e6e82d5c31f"
server_key_iv: "66a03228e60ee6cdcfd23beb9539387d"
server_key_version: "encryptionkey@0"
I20260808 10:45:05.110411 12573 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 10:45:05.113145 12595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:05.114486 12573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:45:05.114619 12573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "b2340a1811ce46fc857fd0ba432de650"
format_stamp: "Formatted at 2026-08-08 10:45:05 on dist-test-slave-twwt"
server_key: "1e909b9440f3b3376d2e7e6e82d5c31f"
server_key_iv: "66a03228e60ee6cdcfd23beb9539387d"
server_key_version: "encryptionkey@0"
I20260808 10:45:05.114782 12573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:05.134960 12573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:05.138448 12573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:05.138693 12573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:05.140228 12573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:05.140285 12573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:05.140355 12573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:05.140396 12573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:45:05 dist-test-slave-twwt krb5kdc[11916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185905, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
I20260808 10:45:05.152257 12573 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:39527
I20260808 10:45:05.152287 12769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:39527 every 8 connection(s)
I20260808 10:45:05.153362 12573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:45:05.155584 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.141248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:46279 (local address 127.31.237.190:45503)
0808 10:45:05.141390 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.141394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.142529 (+ 1135us) server_negotiation.cc:407] Connection header received
0808 10:45:05.143861 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.143865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.143929 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.144037 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:05.145285 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.145844 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.147213 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.147450 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.147565 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.151326 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:05.151334 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:05.151337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:05.151394 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:05.153149 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:05.153807 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:05.153813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:05.153815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:05.153957 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:05.154396 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:05.154400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:05.154401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:05.154720 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:05.154832 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.155334 (+ 502us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.155460 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:45:05.156453 12606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.141802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:45503 (local address 127.31.237.131:46279)
0808 10:45:05.142373 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.142418 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.143618 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.144217 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.144240 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.144631 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:05.145128 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.145142 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.146032 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.146036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.147073 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.147084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.147623 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.147628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.147806 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.148502 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:05.148524 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:05.151082 (+ 2558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:05.153287 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:05.153293 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:05.153310 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:05.153654 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:05.154079 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:05.154083 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:05.154084 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:05.154258 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:05.154854 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:05.154860 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:05.155103 (+ 243us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.155259 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.155533 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":153,"threads_started":1}
I20260808 10:45:05.157733 12770 heartbeater.cc:347] Connected to a master server at 127.31.237.190:45503
I20260808 10:45:05.157914 12770 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:05.158512 12770 heartbeater.cc:511] Master 127.31.237.190:45503 requested a full tablet report, sending...
I20260808 10:45:05.159700 11992 ts_manager.cc:194] Registered new tserver with Master: b2340a1811ce46fc857fd0ba432de650 (127.31.237.131:39527)
I20260808 10:45:05.160593 11992 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:46279
I20260808 10:45:05.162318 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 12573
I20260808 10:45:05.162503 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:45:05.162909 32694 external_mini_cluster.cc:1428] Setting key 34bab1be6ad9991d47045444a8ffe935
I20260808 10:45:05.165290 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:45:05.168165 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.160098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:39681 (local address 127.31.237.190:45503)
0808 10:45:05.160213 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.160217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.160437 (+ 220us) server_negotiation.cc:407] Connection header received
0808 10:45:05.160541 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.160543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.160579 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.160645 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:05.161161 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.161649 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.162874 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.163190 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.163298 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.164544 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:05.164548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:05.164551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:05.164577 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:05.166311 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:05.166794 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:05.166800 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:05.166805 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:05.166913 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:05.167182 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:05.167188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:05.167193 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:05.167468 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:05.167573 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.167822 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.167971 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:45:05.168190 12778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.160017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:45503 (local address 127.31.237.131:39681)
0808 10:45:05.160354 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.160368 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.160464 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.160699 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.160702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.160933 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:05.161061 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.161069 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.161778 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.161781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.162746 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.162757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.163320 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.163323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.163434 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.163980 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:05.163992 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:05.164422 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:05.166427 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:05.166430 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:05.166431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:05.166699 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:05.167009 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:05.167011 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:05.167012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:05.167093 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:05.167567 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:05.167569 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:05.167649 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.167735 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.167913 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":98,"threads_started":1}
Aug 08 10:45:05 dist-test-slave-twwt krb5kdc[11916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185905, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:45:05 dist-test-slave-twwt krb5kdc[11916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:05 dist-test-slave-twwt krb5kdc[11916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:45:05.193053 12074 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:05.186594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39190 (local address 127.31.237.190:45503)
0808 10:45:05.186797 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.186801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.186813 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:45:05.186855 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.186857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.186913 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.187008 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:05.187729 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.188250 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.189114 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.189359 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.189464 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.192894 (+ 3430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39190: BlockingRecv error: recv got EOF from 127.0.0.1:39190 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
Aug 08 10:45:05 dist-test-slave-twwt krb5kdc[11916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185905, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
I20260808 10:45:05.202953 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.194564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39192 (local address 127.31.237.190:45503)
0808 10:45:05.194723 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.194727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.194891 (+ 164us) server_negotiation.cc:407] Connection header received
0808 10:45:05.194987 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.194989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.195030 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.195098 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:05.195801 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.196324 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.197137 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.197336 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.197418 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.199989 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:05.199992 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:05.199995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:05.200018 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:05.201288 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:05.201716 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:05.201719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:05.201720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:05.201791 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:05.202070 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:05.202072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:05.202073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:05.202244 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:05.202403 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.202645 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.202752 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 10:45:05.206051 11992 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:45:05.209302 11992 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:45:05.225091 12804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.218283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:39527 (local address 127.0.0.1:59726)
0808 10:45:05.219579 (+ 1296us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.219634 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.219810 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.220588 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.220591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.220617 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:05.220743 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.220749 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.222197 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.222200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.223414 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.223424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.224447 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.224451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.224585 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.224633 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.224747 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.224879 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1102,"mutex_wait_us":299,"spinlock_wait_cycles":3840,"thread_start_us":77,"threads_started":1}
I20260808 10:45:05.225277 12797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.218323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:37139 (local address 127.0.0.1:59246)
0808 10:45:05.219653 (+ 1330us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.219667 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.219810 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.220588 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.220591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.220622 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:05.220737 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.220744 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.222012 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.222015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.223715 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.223722 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.224791 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.224794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.224899 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.224920 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.224989 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.225130 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1269,"mutex_wait_us":332,"spinlock_wait_cycles":8320,"thread_start_us":113,"threads_started":1}
I20260808 10:45:05.225379 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.219142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59246 (local address 127.31.237.130:37139)
0808 10:45:05.219517 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.219529 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.219748 (+ 219us) server_negotiation.cc:407] Connection header received
0808 10:45:05.220106 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.220110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.220279 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.220420 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:05.220950 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.221846 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.223853 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.224463 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.224601 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.224728 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.225117 (+ 389us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.225230 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":119,"threads_started":1}
I20260808 10:45:05.225672 12800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.218418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59726 (local address 127.31.237.131:39527)
0808 10:45:05.219203 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.219220 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.219723 (+ 503us) server_negotiation.cc:407] Connection header received
0808 10:45:05.220063 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.220069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.220285 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.220413 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:05.220924 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.222072 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.223535 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.224303 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.224452 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.224593 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.225431 (+ 838us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.225538 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":132,"threads_started":1}
I20260808 10:45:05.225934 12798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.218629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:43229 (local address 127.0.0.1:39502)
0808 10:45:05.219808 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.219822 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.219918 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.220398 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.220404 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.220427 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:05.220564 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.220572 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.222010 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.222014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.223704 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.223716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.224742 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.224746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.224838 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.224862 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.224975 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.225070 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1065,"mutex_wait_us":152,"thread_start_us":90,"threads_started":1}
I20260808 10:45:05.226523 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.219102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39502 (local address 127.31.237.129:43229)
0808 10:45:05.219530 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.219537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.219991 (+ 454us) server_negotiation.cc:407] Connection header received
0808 10:45:05.220084 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.220094 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.220254 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.220414 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:05.220664 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.221877 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.223836 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.224621 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.224744 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.224866 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.226267 (+ 1401us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.226379 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":78,"threads_started":1}
I20260808 10:45:05.228626 12446 tablet_service.cc:1467] Processing CreateTablet for tablet 48499b213a9644579cf6389da8f7afd6 (DEFAULT_TABLE table=test-table [id=f822208d4c344f00be86d8cc875b6faa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:05.228626 12670 tablet_service.cc:1467] Processing CreateTablet for tablet 48499b213a9644579cf6389da8f7afd6 (DEFAULT_TABLE table=test-table [id=f822208d4c344f00be86d8cc875b6faa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:05.228971 12210 tablet_service.cc:1467] Processing CreateTablet for tablet 48499b213a9644579cf6389da8f7afd6 (DEFAULT_TABLE table=test-table [id=f822208d4c344f00be86d8cc875b6faa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:05.229830 12446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48499b213a9644579cf6389da8f7afd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:05.229835 12670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48499b213a9644579cf6389da8f7afd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:05.230185 12210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48499b213a9644579cf6389da8f7afd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:05.236502 12812 tablet_bootstrap.cc:492] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1: Bootstrap starting.
I20260808 10:45:05.237854 12815 tablet_bootstrap.cc:492] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650: Bootstrap starting.
I20260808 10:45:05.239197 12813 tablet_bootstrap.cc:492] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae: Bootstrap starting.
I20260808 10:45:05.240397 12812 tablet_bootstrap.cc:654] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:05.240494 12815 tablet_bootstrap.cc:654] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:05.241173 12812 log.cc:824] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:05.241425 12813 tablet_bootstrap.cc:654] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:05.241413 12815 log.cc:824] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:05.242452 12813 log.cc:824] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:05.243613 12815 tablet_bootstrap.cc:492] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650: No bootstrap required, opened a new log
I20260808 10:45:05.243628 12812 tablet_bootstrap.cc:492] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1: No bootstrap required, opened a new log
I20260808 10:45:05.243830 12815 ts_tablet_manager.cc:1397] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:45:05.243844 12812 ts_tablet_manager.cc:1397] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 10:45:05.245620 12813 tablet_bootstrap.cc:492] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae: No bootstrap required, opened a new log
I20260808 10:45:05.245854 12813 ts_tablet_manager.cc:1397] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 10:45:05.247128 12815 raft_consensus.cc:348] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } }
I20260808 10:45:05.247391 12815 raft_consensus.cc:374] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:05.247447 12815 raft_consensus.cc:729] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2340a1811ce46fc857fd0ba432de650, State: Initialized, Role: FOLLOWER
I20260808 10:45:05.247921 12815 consensus_queue.cc:260] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } }
I20260808 10:45:05.248664 12770 heartbeater.cc:503] Master 127.31.237.190:45503 was elected leader, sending a full tablet report...
I20260808 10:45:05.248983 12815 ts_tablet_manager.cc:1428] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 10:45:05.250247 12813 raft_consensus.cc:348] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } }
I20260808 10:45:05.250557 12813 raft_consensus.cc:374] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:05.250643 12813 raft_consensus.cc:729] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6710af7e2a094b979525619766dfb2ae, State: Initialized, Role: FOLLOWER
I20260808 10:45:05.250746 12812 raft_consensus.cc:348] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } }
I20260808 10:45:05.251021 12812 raft_consensus.cc:374] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:05.251080 12812 raft_consensus.cc:729] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c76622db89e413a8b7f4fd0d9fbc0b1, State: Initialized, Role: FOLLOWER
I20260808 10:45:05.251168 12813 consensus_queue.cc:260] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } }
I20260808 10:45:05.251579 12812 consensus_queue.cc:260] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } }
I20260808 10:45:05.252034 12559 heartbeater.cc:503] Master 127.31.237.190:45503 was elected leader, sending a full tablet report...
I20260808 10:45:05.252362 12813 ts_tablet_manager.cc:1428] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20260808 10:45:05.253190 12329 heartbeater.cc:503] Master 127.31.237.190:45503 was elected leader, sending a full tablet report...
I20260808 10:45:05.254724 12812 ts_tablet_manager.cc:1428] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1: Time spent starting tablet: real 0.011s user 0.004s sys 0.004s
I20260808 10:45:05.276463 12826 raft_consensus.cc:482] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:45:05.276741 12826 raft_consensus.cc:504] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } }
I20260808 10:45:05.277921 12826 leader_election.cc:288] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2340a1811ce46fc857fd0ba432de650 (127.31.237.131:39527), 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129:43229)
I20260808 10:45:05.281921 12384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.278186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:39527 (local address 127.31.237.130:57753)
0808 10:45:05.278315 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.278332 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.278441 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.278796 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.278800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.278831 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:05.278957 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.278967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.279749 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.279754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.280807 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.280826 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.281458 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.281462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.281627 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.281665 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.281740 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.281800 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":36}
I20260808 10:45:05.281982 12800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.278349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:57753 (local address 127.31.237.131:39527)
0808 10:45:05.278493 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.278500 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.278516 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:45:05.278573 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.278576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.278659 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.278747 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:05.279116 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.279626 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.280962 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.281339 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.281436 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.281479 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.281779 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.281856 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:45:05.282809 12697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48499b213a9644579cf6389da8f7afd6" candidate_uuid: "6710af7e2a094b979525619766dfb2ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2340a1811ce46fc857fd0ba432de650" is_pre_election: true
I20260808 10:45:05.283020 12829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.278194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:43229 (local address 127.31.237.130:36877)
0808 10:45:05.278952 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.278966 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.279071 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.279363 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.279366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.279382 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:05.279920 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.279930 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.280643 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.280647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.281580 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.281590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.282572 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.282576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.282685 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.282710 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.282787 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.282872 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":328,"thread_start_us":179,"threads_started":1}
I20260808 10:45:05.283035 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.278311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:36877 (local address 127.31.237.129:43229)
0808 10:45:05.278506 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.278510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.279061 (+ 551us) server_negotiation.cc:407] Connection header received
0808 10:45:05.279202 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.279205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.279248 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.279331 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:05.280053 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.280524 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.281697 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.282313 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.282425 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.282462 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.282824 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.282899 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260808 10:45:05.283185 12697 raft_consensus.cc:2454] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6710af7e2a094b979525619766dfb2ae in term 0.
I20260808 10:45:05.283612 12242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48499b213a9644579cf6389da8f7afd6" candidate_uuid: "6710af7e2a094b979525619766dfb2ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" is_pre_election: true
I20260808 10:45:05.283802 12371 leader_election.cc:302] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6710af7e2a094b979525619766dfb2ae, b2340a1811ce46fc857fd0ba432de650; no voters:
I20260808 10:45:05.283929 12242 raft_consensus.cc:2454] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6710af7e2a094b979525619766dfb2ae in term 0.
I20260808 10:45:05.284133 12826 raft_consensus.cc:2781] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:45:05.284243 12826 raft_consensus.cc:482] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:45:05.284302 12826 raft_consensus.cc:3037] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:05.285656 12826 raft_consensus.cc:504] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } }
I20260808 10:45:05.286059 12826 leader_election.cc:288] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [CANDIDATE]: Term 1 election: Requested vote from peers b2340a1811ce46fc857fd0ba432de650 (127.31.237.131:39527), 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129:43229)
I20260808 10:45:05.286409 12242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48499b213a9644579cf6389da8f7afd6" candidate_uuid: "6710af7e2a094b979525619766dfb2ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1"
I20260808 10:45:05.286409 12697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48499b213a9644579cf6389da8f7afd6" candidate_uuid: "6710af7e2a094b979525619766dfb2ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2340a1811ce46fc857fd0ba432de650"
I20260808 10:45:05.286535 12697 raft_consensus.cc:3037] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:05.286535 12242 raft_consensus.cc:3037] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:05.287881 12242 raft_consensus.cc:2454] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6710af7e2a094b979525619766dfb2ae in term 1.
I20260808 10:45:05.287873 12697 raft_consensus.cc:2454] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6710af7e2a094b979525619766dfb2ae in term 1.
I20260808 10:45:05.288192 12371 leader_election.cc:302] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6710af7e2a094b979525619766dfb2ae, b2340a1811ce46fc857fd0ba432de650; no voters:
I20260808 10:45:05.288493 12826 raft_consensus.cc:2781] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:45:05.288736 12826 raft_consensus.cc:686] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [term 1 LEADER]: Becoming Leader. State: Replica: 6710af7e2a094b979525619766dfb2ae, State: Running, Role: LEADER
I20260808 10:45:05.289098 12826 consensus_queue.cc:237] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } }
I20260808 10:45:05.292572 11996 catalog_manager.cc:5495] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 6710af7e2a094b979525619766dfb2ae (127.31.237.130). New cstate: current_term: 1 leader_uuid: "6710af7e2a094b979525619766dfb2ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } health_report { overall_health: UNKNOWN } } }
I20260808 10:45:05.313870 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.309980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59262 (local address 127.31.237.130:37139)
0808 10:45:05.310122 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.310126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.310140 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:45:05.310342 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.310345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.310407 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.310508 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:45:05.310892 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.311439 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.312360 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.312592 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.312678 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.312931 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:45:05.313391 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:45:05.313480 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.313632 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.313686 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:45:05.320679 12242 raft_consensus.cc:1261] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Refusing update from remote peer 6710af7e2a094b979525619766dfb2ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:05.321311 12697 raft_consensus.cc:1261] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Refusing update from remote peer 6710af7e2a094b979525619766dfb2ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:05.321547 12830 consensus_queue.cc:1035] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:05.321957 12826 consensus_queue.cc:1035] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:05.331837 12836 mvcc.cc:205] Tried to move back new op lower bound from 7316217468183728128 to 7316217468067332096. Current Snapshot: MvccSnapshot[applied={T|T < 7316217468183728128}]
I20260808 10:45:05.337705 11996 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:45:05.337893 11996 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:45:05.340768 11996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0: Sending DeleteTablet for 3 replicas of tablet 48499b213a9644579cf6389da8f7afd6
I20260808 10:45:05.341423 12446 tablet_service.cc:1514] Processing DeleteTablet for tablet 48499b213a9644579cf6389da8f7afd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:05 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:59246
I20260808 10:45:05.341552 12670 tablet_service.cc:1514] Processing DeleteTablet for tablet 48499b213a9644579cf6389da8f7afd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:05 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:59726
I20260808 10:45:05.342199 12850 tablet_replica.cc:330] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650: stopping tablet replica
I20260808 10:45:05.342293 12210 tablet_service.cc:1514] Processing DeleteTablet for tablet 48499b213a9644579cf6389da8f7afd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:05 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:39502
I20260808 10:45:05.342458 11992 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:45:05.342728 12209 tablet_service.cc:1514] Processing DeleteTablet for tablet 48499b213a9644579cf6389da8f7afd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:05 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:39502
W20260808 10:45:05.342912 11992 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:45:05.342944 12850 raft_consensus.cc:2229] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:45:05.343003 12848 tablet_replica.cc:330] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae: stopping tablet replica
I20260808 10:45:05.343330 12848 raft_consensus.cc:2229] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:45:05.343475 12850 raft_consensus.cc:2258] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:05.343691 12848 raft_consensus.cc:2258] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:05.345232 12848 ts_tablet_manager.cc:1905] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:05.345230 12850 ts_tablet_manager.cc:1905] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:05.348107 12848 ts_tablet_manager.cc:1918] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:45:05.348209 12848 log.cc:1196] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/48499b213a9644579cf6389da8f7afd6
I20260808 10:45:05.348404 12850 ts_tablet_manager.cc:1918] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:45:05.348534 12848 ts_tablet_manager.cc:1939] T 48499b213a9644579cf6389da8f7afd6 P 6710af7e2a094b979525619766dfb2ae: Deleting consensus metadata
I20260808 10:45:05.348657 12850 log.cc:1196] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/48499b213a9644579cf6389da8f7afd6
I20260808 10:45:05.349069 12850 ts_tablet_manager.cc:1939] T 48499b213a9644579cf6389da8f7afd6 P b2340a1811ce46fc857fd0ba432de650: Deleting consensus metadata
I20260808 10:45:05.349218 12670 tablet_service.cc:1467] Processing CreateTablet for tablet 60cf441ceded4006993da786a366512d (DEFAULT_TABLE table=test-table [id=20ed5ef450a34f19a437dff1e3cbf0ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:05.349514 12670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60cf441ceded4006993da786a366512d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:05.350514 11971 catalog_manager.cc:4841] TS b2340a1811ce46fc857fd0ba432de650 (127.31.237.131:39527): tablet 48499b213a9644579cf6389da8f7afd6 (table test-table [id=f822208d4c344f00be86d8cc875b6faa]) successfully deleted
I20260808 10:45:05.350870 11975 catalog_manager.cc:4841] TS 6710af7e2a094b979525619766dfb2ae (127.31.237.130:37139): tablet 48499b213a9644579cf6389da8f7afd6 (table test-table [id=f822208d4c344f00be86d8cc875b6faa]) successfully deleted
I20260808 10:45:05.351402 12446 tablet_service.cc:1467] Processing CreateTablet for tablet 60cf441ceded4006993da786a366512d (DEFAULT_TABLE table=test-table [id=20ed5ef450a34f19a437dff1e3cbf0ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:05.351679 12446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60cf441ceded4006993da786a366512d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:05.351708 12209 tablet_service.cc:1467] Processing CreateTablet for tablet 60cf441ceded4006993da786a366512d (DEFAULT_TABLE table=test-table [id=20ed5ef450a34f19a437dff1e3cbf0ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:05.351977 12209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60cf441ceded4006993da786a366512d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:05.352465 12815 tablet_bootstrap.cc:492] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650: Bootstrap starting.
I20260808 10:45:05.353518 12815 tablet_bootstrap.cc:654] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:05.354154 12813 tablet_bootstrap.cc:492] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae: Bootstrap starting.
I20260808 10:45:05.354465 12812 tablet_bootstrap.cc:492] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1: Bootstrap starting.
I20260808 10:45:05.355034 12851 tablet_replica.cc:330] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1: stopping tablet replica
I20260808 10:45:05.355173 12813 tablet_bootstrap.cc:654] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:05.355469 12812 tablet_bootstrap.cc:654] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:05.355588 12851 raft_consensus.cc:2229] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:45:05.355997 12851 raft_consensus.cc:2258] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:05.356451 12813 tablet_bootstrap.cc:492] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae: No bootstrap required, opened a new log
I20260808 10:45:05.356537 12813 ts_tablet_manager.cc:1397] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:45:05.357084 12815 tablet_bootstrap.cc:492] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650: No bootstrap required, opened a new log
I20260808 10:45:05.357093 12813 raft_consensus.cc:348] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.357177 12815 ts_tablet_manager.cc:1397] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 10:45:05.357219 12813 raft_consensus.cc:374] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:05.357265 12813 raft_consensus.cc:729] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6710af7e2a094b979525619766dfb2ae, State: Initialized, Role: FOLLOWER
I20260808 10:45:05.357438 12813 consensus_queue.cc:260] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.357750 12813 ts_tablet_manager.cc:1428] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260808 10:45:05.357736 12815 raft_consensus.cc:348] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.357878 12815 raft_consensus.cc:374] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:05.357920 12815 raft_consensus.cc:729] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2340a1811ce46fc857fd0ba432de650, State: Initialized, Role: FOLLOWER
I20260808 10:45:05.358086 12815 consensus_queue.cc:260] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.358340 12815 ts_tablet_manager.cc:1428] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 10:45:05.359961 12851 ts_tablet_manager.cc:1905] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:05.362165 12812 tablet_bootstrap.cc:492] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1: No bootstrap required, opened a new log
I20260808 10:45:05.362258 12812 ts_tablet_manager.cc:1397] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 10:45:05.362880 12851 ts_tablet_manager.cc:1918] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:45:05.363029 12851 log.cc:1196] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/wal/wals/48499b213a9644579cf6389da8f7afd6
I20260808 10:45:05.363575 12851 ts_tablet_manager.cc:1939] T 48499b213a9644579cf6389da8f7afd6 P 1c76622db89e413a8b7f4fd0d9fbc0b1: Deleting consensus metadata
I20260808 10:45:05.363541 12812 raft_consensus.cc:348] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.363718 12812 raft_consensus.cc:374] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:05.363766 12812 raft_consensus.cc:729] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c76622db89e413a8b7f4fd0d9fbc0b1, State: Initialized, Role: FOLLOWER
I20260808 10:45:05.363893 12812 consensus_queue.cc:260] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.364141 12812 ts_tablet_manager.cc:1428] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:45:05.365147 11974 catalog_manager.cc:4841] TS 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129:43229): tablet 48499b213a9644579cf6389da8f7afd6 (table test-table [id=f822208d4c344f00be86d8cc875b6faa]) successfully deleted
W20260808 10:45:05.365952 11974 catalog_manager.cc:4808] TS 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129:43229): delete failed for tablet 48499b213a9644579cf6389da8f7afd6 because the tablet was not found. No further retry: Not found: Tablet not found: 48499b213a9644579cf6389da8f7afd6
W20260808 10:45:05.404469 12773 tablet.cc:2369] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:45:05.417651 12561 tablet.cc:2369] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:45:05.442492 12330 tablet.cc:2369] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:45:05.458784 12824 raft_consensus.cc:482] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:45:05.458930 12824 raft_consensus.cc:504] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.460035 12824 leader_election.cc:288] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129:43229), 6710af7e2a094b979525619766dfb2ae (127.31.237.130:37139)
I20260808 10:45:05.464898 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.460734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:35191 (local address 127.31.237.129:43229)
0808 10:45:05.460870 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.460875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.460912 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:45:05.460971 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.460974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.461026 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.461109 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:05.461442 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.461943 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.463667 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.464330 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.464474 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.464536 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.464701 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.464781 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:45:05.464916 12855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.460301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:43229 (local address 127.31.237.131:35191)
0808 10:45:05.460722 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.460736 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.460840 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.461139 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.461144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.461158 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:05.461303 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.461310 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.462069 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.462074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.463521 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.463533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.464467 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.464470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.464555 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.464578 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.464677 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.464749 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":29,"thread_start_us":89,"threads_started":1}
I20260808 10:45:05.465915 12606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.460282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:37139 (local address 127.31.237.131:54303)
0808 10:45:05.460479 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.460497 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.460593 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.461018 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.461020 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.461035 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:05.461214 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.461222 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.462221 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.462224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.463521 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.463533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.465484 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.465488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.465605 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.465628 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.465713 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.465785 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":80}
I20260808 10:45:05.465866 12242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60cf441ceded4006993da786a366512d" candidate_uuid: "b2340a1811ce46fc857fd0ba432de650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" is_pre_election: true
I20260808 10:45:05.465966 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.460427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:54303 (local address 127.31.237.130:37139)
0808 10:45:05.460557 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.460561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.460577 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:45:05.460744 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.460749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.460812 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.460894 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:05.461360 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.462098 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.464604 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.465224 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.465366 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.465419 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.465747 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.465828 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:45:05.466034 12242 raft_consensus.cc:2454] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2340a1811ce46fc857fd0ba432de650 in term 0.
I20260808 10:45:05.466506 12597 leader_election.cc:302] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c76622db89e413a8b7f4fd0d9fbc0b1, b2340a1811ce46fc857fd0ba432de650; no voters:
I20260808 10:45:05.466652 12487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60cf441ceded4006993da786a366512d" candidate_uuid: "b2340a1811ce46fc857fd0ba432de650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6710af7e2a094b979525619766dfb2ae" is_pre_election: true
I20260808 10:45:05.466825 12824 raft_consensus.cc:2781] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:45:05.466881 12824 raft_consensus.cc:482] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:45:05.466928 12824 raft_consensus.cc:3037] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:05.466900 12487 raft_consensus.cc:2454] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2340a1811ce46fc857fd0ba432de650 in term 0.
I20260808 10:45:05.468086 12824 raft_consensus.cc:504] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.468434 12824 leader_election.cc:288] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [CANDIDATE]: Term 1 election: Requested vote from peers 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129:43229), 6710af7e2a094b979525619766dfb2ae (127.31.237.130:37139)
I20260808 10:45:05.468763 12487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60cf441ceded4006993da786a366512d" candidate_uuid: "b2340a1811ce46fc857fd0ba432de650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6710af7e2a094b979525619766dfb2ae"
I20260808 10:45:05.468830 12242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60cf441ceded4006993da786a366512d" candidate_uuid: "b2340a1811ce46fc857fd0ba432de650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1"
I20260808 10:45:05.468894 12487 raft_consensus.cc:3037] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:05.468966 12242 raft_consensus.cc:3037] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:05.469857 12487 raft_consensus.cc:2454] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2340a1811ce46fc857fd0ba432de650 in term 1.
I20260808 10:45:05.470147 12598 leader_election.cc:302] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6710af7e2a094b979525619766dfb2ae, b2340a1811ce46fc857fd0ba432de650; no voters:
I20260808 10:45:05.470224 12242 raft_consensus.cc:2454] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2340a1811ce46fc857fd0ba432de650 in term 1.
I20260808 10:45:05.470351 12824 raft_consensus.cc:2781] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:45:05.470562 12824 raft_consensus.cc:686] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [term 1 LEADER]: Becoming Leader. State: Replica: b2340a1811ce46fc857fd0ba432de650, State: Running, Role: LEADER
I20260808 10:45:05.470861 12824 consensus_queue.cc:237] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.473325 11996 catalog_manager.cc:5495] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2340a1811ce46fc857fd0ba432de650 (127.31.237.131). New cstate: current_term: 1 leader_uuid: "b2340a1811ce46fc857fd0ba432de650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } health_report { overall_health: HEALTHY } } }
Aug 08 10:45:05 dist-test-slave-twwt krb5kdc[11916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185904, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.190@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
I20260808 10:45:05.509713 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.500429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:45503 (local address 127.0.0.1:39204)
0808 10:45:05.500740 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.500757 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.500882 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.501136 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.501140 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.501658 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:05.501785 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.501792 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.502536 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.502539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.503402 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.503415 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.503898 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.503902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.503988 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.504590 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:05.504604 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:05.506311 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:05.508089 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:05.508099 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:05.508105 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:05.508371 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:05.508648 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:05.508651 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:05.508653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:05.508745 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:05.509158 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:05.509164 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:05.509304 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.509409 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.509543 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":120,"threads_started":1}
I20260808 10:45:05.509729 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.500490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39204 (local address 127.31.237.190:45503)
0808 10:45:05.500639 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.500643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.500840 (+ 197us) server_negotiation.cc:407] Connection header received
0808 10:45:05.500973 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.500976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.501018 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.501090 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:05.501915 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.502428 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.503523 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.503781 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.503861 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.506468 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:05.506471 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:05.506473 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:05.506495 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:05.507967 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:05.508461 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:05.508464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:05.508465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:05.508541 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:05.508847 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:05.508849 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:05.508850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:05.509037 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:05.509113 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.509468 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.509603 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":62}
I20260808 10:45:05.513724 11996 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:39204:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:45:05.514406 11996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:45:05.519605 12209 tablet_service.cc:1467] Processing CreateTablet for tablet 0a68190ccfe94676bcaa1edfd6e9e1ff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=532c07657c2a498f9ea1174c39778bef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:45:05.519728 12446 tablet_service.cc:1467] Processing CreateTablet for tablet 0a68190ccfe94676bcaa1edfd6e9e1ff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=532c07657c2a498f9ea1174c39778bef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:45:05.519796 12670 tablet_service.cc:1467] Processing CreateTablet for tablet 0a68190ccfe94676bcaa1edfd6e9e1ff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=532c07657c2a498f9ea1174c39778bef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:45:05.520047 12209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a68190ccfe94676bcaa1edfd6e9e1ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:05.520155 12670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a68190ccfe94676bcaa1edfd6e9e1ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:05.520155 12446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a68190ccfe94676bcaa1edfd6e9e1ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:05.522890 12812 tablet_bootstrap.cc:492] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1: Bootstrap starting.
I20260808 10:45:05.522964 12813 tablet_bootstrap.cc:492] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae: Bootstrap starting.
I20260808 10:45:05.523931 12813 tablet_bootstrap.cc:654] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:05.523941 12815 tablet_bootstrap.cc:492] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650: Bootstrap starting.
I20260808 10:45:05.523967 12812 tablet_bootstrap.cc:654] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:05.524966 12815 tablet_bootstrap.cc:654] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:05.525046 12813 tablet_bootstrap.cc:492] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae: No bootstrap required, opened a new log
I20260808 10:45:05.525107 12813 ts_tablet_manager.cc:1397] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:45:05.525178 12812 tablet_bootstrap.cc:492] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1: No bootstrap required, opened a new log
I20260808 10:45:05.525266 12812 ts_tablet_manager.cc:1397] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 10:45:05.525426 12813 raft_consensus.cc:348] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.525512 12813 raft_consensus.cc:374] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:05.525534 12813 raft_consensus.cc:729] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6710af7e2a094b979525619766dfb2ae, State: Initialized, Role: FOLLOWER
I20260808 10:45:05.525676 12813 consensus_queue.cc:260] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.525731 12812 raft_consensus.cc:348] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.525846 12812 raft_consensus.cc:374] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:05.525904 12812 raft_consensus.cc:729] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c76622db89e413a8b7f4fd0d9fbc0b1, State: Initialized, Role: FOLLOWER
I20260808 10:45:05.525959 12813 ts_tablet_manager.cc:1428] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260808 10:45:05.525905 12844 tablet_replica.cc:439] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } }
I20260808 10:45:05.526042 12844 tablet_replica.cc:442] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:45:05.526042 12812 consensus_queue.cc:260] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.526324 12812 ts_tablet_manager.cc:1428] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:45:05.526283 12828 tablet_replica.cc:439] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } }
I20260808 10:45:05.526432 12828 tablet_replica.cc:442] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:45:05.527190 12815 tablet_bootstrap.cc:492] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650: No bootstrap required, opened a new log
I20260808 10:45:05.527292 12815 ts_tablet_manager.cc:1397] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:45:05.527792 12815 raft_consensus.cc:348] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.527921 12815 raft_consensus.cc:374] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:05.527966 12815 raft_consensus.cc:729] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2340a1811ce46fc857fd0ba432de650, State: Initialized, Role: FOLLOWER
I20260808 10:45:05.528095 12815 consensus_queue.cc:260] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.528405 12815 ts_tablet_manager.cc:1428] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:45:05.528339 12824 tablet_replica.cc:439] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } }
I20260808 10:45:05.528496 12824 tablet_replica.cc:442] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:45:05.641749 12828 raft_consensus.cc:482] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:45:05.641922 12828 raft_consensus.cc:504] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.643139 12828 leader_election.cc:288] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6710af7e2a094b979525619766dfb2ae (127.31.237.130:37139), b2340a1811ce46fc857fd0ba432de650 (127.31.237.131:39527)
I20260808 10:45:05.647320 12800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.643569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:47247 (local address 127.31.237.131:39527)
0808 10:45:05.643832 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.643838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.643879 (+ 41us) server_negotiation.cc:407] Connection header received
0808 10:45:05.644106 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.644111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.644169 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.644224 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:05.644541 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.645075 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.646222 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.646664 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.646759 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.646811 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.647109 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.647183 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260808 10:45:05.647337 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.643526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:35627 (local address 127.31.237.130:37139)
0808 10:45:05.643673 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.643678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.644007 (+ 329us) server_negotiation.cc:407] Connection header received
0808 10:45:05.644166 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.644171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.644231 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.644308 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:05.644625 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.645242 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.646294 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.646775 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.646892 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.646955 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.647147 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.647229 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:45:05.647593 12885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.643470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:37139 (local address 127.31.237.129:35627)
0808 10:45:05.643890 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.643905 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.644011 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.644353 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.644357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.644378 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:05.644503 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.644510 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.645364 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.645366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.646135 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.646173 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.646906 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.646909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.646999 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.647017 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.647113 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.647183 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":41,"thread_start_us":144,"threads_started":1}
I20260808 10:45:05.647352 12883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.643467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:39527 (local address 127.31.237.129:47247)
0808 10:45:05.643761 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.643777 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.643949 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.644243 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.644248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.644269 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:05.644413 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.644420 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.645180 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.645184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.646083 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.646094 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.646780 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.646784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.646954 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.646987 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.647063 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.647161 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":26,"thread_start_us":77,"threads_started":1}
I20260808 10:45:05.648068 12487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a68190ccfe94676bcaa1edfd6e9e1ff" candidate_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6710af7e2a094b979525619766dfb2ae" is_pre_election: true
I20260808 10:45:05.648236 12487 raft_consensus.cc:2454] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c76622db89e413a8b7f4fd0d9fbc0b1 in term 0.
I20260808 10:45:05.648254 12697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a68190ccfe94676bcaa1edfd6e9e1ff" candidate_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2340a1811ce46fc857fd0ba432de650" is_pre_election: true
I20260808 10:45:05.648406 12697 raft_consensus.cc:2454] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c76622db89e413a8b7f4fd0d9fbc0b1 in term 0.
I20260808 10:45:05.648659 12138 leader_election.cc:302] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c76622db89e413a8b7f4fd0d9fbc0b1, 6710af7e2a094b979525619766dfb2ae; no voters:
I20260808 10:45:05.648957 12828 raft_consensus.cc:2781] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:45:05.649058 12828 raft_consensus.cc:482] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:45:05.649118 12828 raft_consensus.cc:3037] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:05.650163 12828 raft_consensus.cc:504] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.650554 12828 leader_election.cc:288] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [CANDIDATE]: Term 1 election: Requested vote from peers 6710af7e2a094b979525619766dfb2ae (127.31.237.130:37139), b2340a1811ce46fc857fd0ba432de650 (127.31.237.131:39527)
I20260808 10:45:05.650907 12487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a68190ccfe94676bcaa1edfd6e9e1ff" candidate_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6710af7e2a094b979525619766dfb2ae"
I20260808 10:45:05.650969 12697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a68190ccfe94676bcaa1edfd6e9e1ff" candidate_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2340a1811ce46fc857fd0ba432de650"
I20260808 10:45:05.651050 12487 raft_consensus.cc:3037] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:05.651088 12697 raft_consensus.cc:3037] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:05.651974 12487 raft_consensus.cc:2454] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c76622db89e413a8b7f4fd0d9fbc0b1 in term 1.
I20260808 10:45:05.651997 12697 raft_consensus.cc:2454] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c76622db89e413a8b7f4fd0d9fbc0b1 in term 1.
I20260808 10:45:05.652294 12134 leader_election.cc:302] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c76622db89e413a8b7f4fd0d9fbc0b1, b2340a1811ce46fc857fd0ba432de650; no voters:
I20260808 10:45:05.652485 12828 raft_consensus.cc:2781] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:45:05.652711 12828 raft_consensus.cc:686] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 LEADER]: Becoming Leader. State: Replica: 1c76622db89e413a8b7f4fd0d9fbc0b1, State: Running, Role: LEADER
I20260808 10:45:05.652990 12828 consensus_queue.cc:237] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.654400 12886 tablet_replica.cc:439] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1: TxnStatusTablet state changed. Reason: New leader 1c76622db89e413a8b7f4fd0d9fbc0b1. Latest consensus state: current_term: 1 leader_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } }
I20260808 10:45:05.654521 12886 tablet_replica.cc:442] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:45:05.654831 12892 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:45:05.655015 12892 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 10:45:05.655550 11992 catalog_manager.cc:5495] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129). New cstate: current_term: 1 leader_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } health_report { overall_health: UNKNOWN } } }
I20260808 10:45:05.688479 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.684041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39506 (local address 127.31.237.129:43229)
0808 10:45:05.684236 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.684240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.684279 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:45:05.684536 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.684543 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.684606 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.684692 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:45:05.685009 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.685650 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.686918 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.687136 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.687214 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.687567 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:45:05.688037 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:45:05.688124 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.688282 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.688339 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260808 10:45:05.688553 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.683996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:43229 (local address 127.0.0.1:39506)
0808 10:45:05.684184 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.684201 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.684353 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.684730 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.684733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.684742 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:45:05.684880 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.684887 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.685754 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.685758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.686773 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.686785 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.687243 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.687247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.687350 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.687411 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:45:05.688141 (+ 730us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:45:05.688157 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.688259 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.688403 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 10:45:05.692153 12697 raft_consensus.cc:1261] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Refusing update from remote peer 1c76622db89e413a8b7f4fd0d9fbc0b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:05.692342 12487 raft_consensus.cc:1261] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Refusing update from remote peer 1c76622db89e413a8b7f4fd0d9fbc0b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:05.692962 12828 consensus_queue.cc:1035] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:05.693179 12886 consensus_queue.cc:1035] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:05.694972 12824 tablet_replica.cc:439] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650: TxnStatusTablet state changed. Reason: New leader 1c76622db89e413a8b7f4fd0d9fbc0b1. Latest consensus state: current_term: 1 leader_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } }
I20260808 10:45:05.695127 12824 tablet_replica.cc:442] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:45:05.695173 12844 tablet_replica.cc:439] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae: TxnStatusTablet state changed. Reason: New leader 1c76622db89e413a8b7f4fd0d9fbc0b1. Latest consensus state: current_term: 1 leader_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } }
I20260808 10:45:05.695291 12844 tablet_replica.cc:442] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:45:05.697320 12834 mvcc.cc:205] Tried to move back new op lower bound from 7316217469709254656 to 7316217469556973568. Current Snapshot: MvccSnapshot[applied={T|T < 7316217469709254656}]
I20260808 10:45:05.697748 12886 tablet_replica.cc:439] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } }
I20260808 10:45:05.697860 12886 tablet_replica.cc:442] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:45:05.699138 12833 mvcc.cc:205] Tried to move back new op lower bound from 7316217469709254656 to 7316217469556973568. Current Snapshot: MvccSnapshot[applied={T|T < 7316217469709254656}]
I20260808 10:45:05.699317 12844 tablet_replica.cc:439] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } }
I20260808 10:45:05.699414 12844 tablet_replica.cc:442] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 6710af7e2a094b979525619766dfb2ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:45:05.700047 12828 tablet_replica.cc:439] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } }
I20260808 10:45:05.700156 12828 tablet_replica.cc:442] T 0a68190ccfe94676bcaa1edfd6e9e1ff P 1c76622db89e413a8b7f4fd0d9fbc0b1: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:45:05.701114 12824 tablet_replica.cc:439] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } } }
I20260808 10:45:05.701228 12824 tablet_replica.cc:442] T 0a68190ccfe94676bcaa1edfd6e9e1ff P b2340a1811ce46fc857fd0ba432de650: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:45:05.707680 12800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.703429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59732 (local address 127.31.237.131:39527)
0808 10:45:05.703536 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.703541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.703554 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:45:05.703762 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.703765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.703851 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.703971 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:45:05.704357 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.705052 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.706135 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.706377 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.706495 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.706858 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:45:05.707295 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:45:05.707426 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.707492 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.707550 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:45:05.718659 12807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.714937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:60093 (local address 127.31.237.129:43229)
0808 10:45:05.715064 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.715069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.715346 (+ 277us) server_negotiation.cc:407] Connection header received
0808 10:45:05.715518 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.715521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.715579 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.715662 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:45:05.715919 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.716447 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.717311 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.717544 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.717658 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.718019 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:45:05.718147 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:45:05.718303 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.718435 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.718486 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:45:05.718672 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.714873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:43229 (local address 127.31.237.131:60093)
0808 10:45:05.715254 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.715267 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.715401 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.715673 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.715676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.715684 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:45:05.715810 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.715816 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.716560 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.716565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.717181 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.717189 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.717674 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.717678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.717824 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.717872 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:45:05.718290 (+ 418us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:45:05.718298 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.718399 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.718487 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":123,"threads_started":1}
I20260808 10:45:05.726689 12487 raft_consensus.cc:1261] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Refusing update from remote peer b2340a1811ce46fc857fd0ba432de650: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:05.726706 12242 raft_consensus.cc:1261] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Refusing update from remote peer b2340a1811ce46fc857fd0ba432de650: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:05.727183 12824 consensus_queue.cc:1035] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:05.727380 12858 consensus_queue.cc:1035] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:05.752316 12800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.747827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:59725 (local address 127.31.237.131:39527)
0808 10:45:05.747967 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.747972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.748195 (+ 223us) server_negotiation.cc:407] Connection header received
0808 10:45:05.748388 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.748391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.748446 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.748546 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:45:05.748947 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.749745 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.751065 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.751320 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.751415 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.751800 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:45:05.751896 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:45:05.751941 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.752103 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.752168 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:45:05.752748 12914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.747693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:39527 (local address 127.31.237.129:59725)
0808 10:45:05.748112 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:05.748126 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:05.748218 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:05.748669 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:05.748673 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:05.748681 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:45:05.748823 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.748829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.749887 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.749890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.750919 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:05.750930 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.751448 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.751452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:05.751568 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.751633 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:45:05.752004 (+ 371us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:45:05.752010 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:45:05.752497 (+ 487us) client_negotiation.cc:240] Negotiation successful
0808 10:45:05.752591 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":176,"threads_started":1}
I20260808 10:45:05.782407 11992 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:45:05.782656 11992 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:45:05.785194 11992 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0: Sending DeleteTablet for 3 replicas of tablet 60cf441ceded4006993da786a366512d
I20260808 10:45:05.785900 12446 tablet_service.cc:1514] Processing DeleteTablet for tablet 60cf441ceded4006993da786a366512d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:05 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:59246
I20260808 10:45:05.786010 12670 tablet_service.cc:1514] Processing DeleteTablet for tablet 60cf441ceded4006993da786a366512d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:05 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:59726
I20260808 10:45:05.786109 12848 tablet_replica.cc:330] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae: stopping tablet replica
I20260808 10:45:05.786230 12848 raft_consensus.cc:2229] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:45:05.786403 12848 raft_consensus.cc:2258] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:05.786597 12850 tablet_replica.cc:330] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650: stopping tablet replica
I20260808 10:45:05.786739 12850 raft_consensus.cc:2229] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:45:05.787081 12850 raft_consensus.cc:2258] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:05.788239 12848 ts_tablet_manager.cc:1905] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:05.791090 12209 tablet_service.cc:1514] Processing DeleteTablet for tablet 60cf441ceded4006993da786a366512d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:05 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:39502
I20260808 10:45:05.791318 12851 tablet_replica.cc:330] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1: stopping tablet replica
I20260808 10:45:05.791435 12851 raft_consensus.cc:2229] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:45:05.791531 12850 ts_tablet_manager.cc:1905] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:05.791568 12851 raft_consensus.cc:2258] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:05.792192 12848 ts_tablet_manager.cc:1918] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:45:05.792330 12848 log.cc:1196] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/60cf441ceded4006993da786a366512d
I20260808 10:45:05.792708 12848 ts_tablet_manager.cc:1939] T 60cf441ceded4006993da786a366512d P 6710af7e2a094b979525619766dfb2ae: Deleting consensus metadata
I20260808 10:45:05.792356 12851 ts_tablet_manager.cc:1905] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:05.793601 11975 catalog_manager.cc:4841] TS 6710af7e2a094b979525619766dfb2ae (127.31.237.130:37139): tablet 60cf441ceded4006993da786a366512d (table test-table [id=20ed5ef450a34f19a437dff1e3cbf0ab]) successfully deleted
I20260808 10:45:05.794690 12850 ts_tablet_manager.cc:1918] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:45:05.794790 12850 log.cc:1196] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/60cf441ceded4006993da786a366512d
I20260808 10:45:05.795167 12850 ts_tablet_manager.cc:1939] T 60cf441ceded4006993da786a366512d P b2340a1811ce46fc857fd0ba432de650: Deleting consensus metadata
I20260808 10:45:05.795904 11971 catalog_manager.cc:4841] TS b2340a1811ce46fc857fd0ba432de650 (127.31.237.131:39527): tablet 60cf441ceded4006993da786a366512d (table test-table [id=20ed5ef450a34f19a437dff1e3cbf0ab]) successfully deleted
I20260808 10:45:05.796360 12851 ts_tablet_manager.cc:1918] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:45:05.796425 12851 log.cc:1196] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/wal/wals/60cf441ceded4006993da786a366512d
I20260808 10:45:05.796669 12851 ts_tablet_manager.cc:1939] T 60cf441ceded4006993da786a366512d P 1c76622db89e413a8b7f4fd0d9fbc0b1: Deleting consensus metadata
I20260808 10:45:05.797322 11974 catalog_manager.cc:4841] TS 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129:43229): tablet 60cf441ceded4006993da786a366512d (table test-table [id=20ed5ef450a34f19a437dff1e3cbf0ab]) successfully deleted
Aug 08 10:45:05 dist-test-slave-twwt krb5kdc[11916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:45:05 dist-test-slave-twwt krb5kdc[11916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:05 dist-test-slave-twwt krb5kdc[11916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:45:05.831018 12074 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:05.820864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39218 (local address 127.31.237.190:45503)
0808 10:45:05.820999 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.821004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.822938 (+ 1934us) server_negotiation.cc:407] Connection header received
0808 10:45:05.823099 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.823103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.823167 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.823262 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:05.824181 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.825022 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.826275 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.826690 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.826809 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.830876 (+ 4067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39218: BlockingRecv error: recv got EOF from 127.0.0.1:39218 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 10:45:05 dist-test-slave-twwt krb5kdc[11916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM
I20260808 10:45:05.844481 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:05.833583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39226 (local address 127.31.237.190:45503)
0808 10:45:05.833699 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:05.833703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:05.834152 (+ 449us) server_negotiation.cc:407] Connection header received
0808 10:45:05.834266 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:05.834269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:05.834325 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:05.834413 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:05.835227 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.836045 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.837210 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:05.837553 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:05.837659 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:05.840737 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:05.840743 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:05.840749 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:05.840782 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:05.842529 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:05.843060 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:05.843064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:05.843065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:05.843174 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:05.843529 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:05.843532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:05.843534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:05.843748 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:05.843876 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:05.844172 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 10:45:05.844295 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:45:05.847287 11992 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:45:05.847764 11992 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:45:05.852579 12209 tablet_service.cc:1467] Processing CreateTablet for tablet 91b6ef48e1b94812ac545e99dd49ba4b (DEFAULT_TABLE table=test-table [id=e46ef80e7d8d4472b58a15e98d151b07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:05.852739 12446 tablet_service.cc:1467] Processing CreateTablet for tablet 91b6ef48e1b94812ac545e99dd49ba4b (DEFAULT_TABLE table=test-table [id=e46ef80e7d8d4472b58a15e98d151b07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:05.852927 12209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91b6ef48e1b94812ac545e99dd49ba4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:05.853050 12446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91b6ef48e1b94812ac545e99dd49ba4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:05.853852 12670 tablet_service.cc:1467] Processing CreateTablet for tablet 91b6ef48e1b94812ac545e99dd49ba4b (DEFAULT_TABLE table=test-table [id=e46ef80e7d8d4472b58a15e98d151b07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:05.854132 12670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91b6ef48e1b94812ac545e99dd49ba4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:05.855501 12812 tablet_bootstrap.cc:492] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1: Bootstrap starting.
I20260808 10:45:05.855535 12813 tablet_bootstrap.cc:492] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae: Bootstrap starting.
I20260808 10:45:05.856352 12812 tablet_bootstrap.cc:654] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:05.856460 12813 tablet_bootstrap.cc:654] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:05.856638 12815 tablet_bootstrap.cc:492] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650: Bootstrap starting.
I20260808 10:45:05.857609 12812 tablet_bootstrap.cc:492] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1: No bootstrap required, opened a new log
I20260808 10:45:05.857630 12815 tablet_bootstrap.cc:654] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:05.857684 12812 ts_tablet_manager.cc:1397] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:45:05.858083 12812 raft_consensus.cc:348] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.858180 12813 tablet_bootstrap.cc:492] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae: No bootstrap required, opened a new log
I20260808 10:45:05.858191 12812 raft_consensus.cc:374] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:05.858222 12812 raft_consensus.cc:729] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c76622db89e413a8b7f4fd0d9fbc0b1, State: Initialized, Role: FOLLOWER
I20260808 10:45:05.858264 12813 ts_tablet_manager.cc:1397] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 10:45:05.858342 12812 consensus_queue.cc:260] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.858585 12812 ts_tablet_manager.cc:1428] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:45:05.858741 12813 raft_consensus.cc:348] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.858857 12813 raft_consensus.cc:374] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:05.858891 12813 raft_consensus.cc:729] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6710af7e2a094b979525619766dfb2ae, State: Initialized, Role: FOLLOWER
I20260808 10:45:05.859015 12813 consensus_queue.cc:260] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.859319 12813 ts_tablet_manager.cc:1428] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:45:05.859961 12815 tablet_bootstrap.cc:492] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650: No bootstrap required, opened a new log
I20260808 10:45:05.860045 12815 ts_tablet_manager.cc:1397] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:45:05.860476 12815 raft_consensus.cc:348] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.860587 12815 raft_consensus.cc:374] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:05.860617 12815 raft_consensus.cc:729] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2340a1811ce46fc857fd0ba432de650, State: Initialized, Role: FOLLOWER
I20260808 10:45:05.860740 12815 consensus_queue.cc:260] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:05.860997 12815 ts_tablet_manager.cc:1428] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:45:06.167336 12912 raft_consensus.cc:482] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:45:06.167480 12912 raft_consensus.cc:504] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:06.167953 12912 leader_election.cc:288] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129:43229), 6710af7e2a094b979525619766dfb2ae (127.31.237.130:37139)
I20260808 10:45:06.168445 12242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91b6ef48e1b94812ac545e99dd49ba4b" candidate_uuid: "b2340a1811ce46fc857fd0ba432de650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" is_pre_election: true
I20260808 10:45:06.168466 12487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91b6ef48e1b94812ac545e99dd49ba4b" candidate_uuid: "b2340a1811ce46fc857fd0ba432de650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6710af7e2a094b979525619766dfb2ae" is_pre_election: true
I20260808 10:45:06.168615 12242 raft_consensus.cc:2454] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2340a1811ce46fc857fd0ba432de650 in term 0.
I20260808 10:45:06.168624 12487 raft_consensus.cc:2454] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2340a1811ce46fc857fd0ba432de650 in term 0.
I20260808 10:45:06.168918 12597 leader_election.cc:302] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c76622db89e413a8b7f4fd0d9fbc0b1, b2340a1811ce46fc857fd0ba432de650; no voters:
I20260808 10:45:06.169104 12912 raft_consensus.cc:2781] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:45:06.169170 12912 raft_consensus.cc:482] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:45:06.169205 12912 raft_consensus.cc:3037] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:06.170450 12912 raft_consensus.cc:504] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:06.170817 12912 leader_election.cc:288] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [CANDIDATE]: Term 1 election: Requested vote from peers 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129:43229), 6710af7e2a094b979525619766dfb2ae (127.31.237.130:37139)
I20260808 10:45:06.171190 12242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91b6ef48e1b94812ac545e99dd49ba4b" candidate_uuid: "b2340a1811ce46fc857fd0ba432de650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1"
I20260808 10:45:06.171232 12487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91b6ef48e1b94812ac545e99dd49ba4b" candidate_uuid: "b2340a1811ce46fc857fd0ba432de650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6710af7e2a094b979525619766dfb2ae"
I20260808 10:45:06.171315 12242 raft_consensus.cc:3037] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:06.171360 12487 raft_consensus.cc:3037] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:06.172544 12242 raft_consensus.cc:2454] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2340a1811ce46fc857fd0ba432de650 in term 1.
I20260808 10:45:06.172544 12487 raft_consensus.cc:2454] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2340a1811ce46fc857fd0ba432de650 in term 1.
I20260808 10:45:06.172869 12598 leader_election.cc:302] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6710af7e2a094b979525619766dfb2ae, b2340a1811ce46fc857fd0ba432de650; no voters:
I20260808 10:45:06.173050 12912 raft_consensus.cc:2781] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:45:06.173174 12912 raft_consensus.cc:686] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [term 1 LEADER]: Becoming Leader. State: Replica: b2340a1811ce46fc857fd0ba432de650, State: Running, Role: LEADER
I20260808 10:45:06.173329 12912 consensus_queue.cc:237] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:06.175216 11996 catalog_manager.cc:5495] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2340a1811ce46fc857fd0ba432de650 (127.31.237.131). New cstate: current_term: 1 leader_uuid: "b2340a1811ce46fc857fd0ba432de650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } health_report { overall_health: HEALTHY } } }
I20260808 10:45:06.227751 12800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:06.224489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.31.237.131:39527)
0808 10:45:06.224635 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:06.224640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:06.224655 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:45:06.224812 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:06.224815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:06.224869 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:06.224956 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:45:06.225222 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:06.225738 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:06.226633 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:06.226853 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:06.226969 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:06.227238 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:45:06.227310 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:45:06.227408 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:06.227542 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 10:45:06.227598 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:45:06.229987 12242 raft_consensus.cc:1261] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Refusing update from remote peer b2340a1811ce46fc857fd0ba432de650: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:06.230046 12487 raft_consensus.cc:1261] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Refusing update from remote peer b2340a1811ce46fc857fd0ba432de650: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:06.230521 12912 consensus_queue.cc:1035] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:06.230757 12858 consensus_queue.cc:1035] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:06.238144 11996 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:45:06.238349 11996 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:45:06.240440 11996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0: Sending DeleteTablet for 3 replicas of tablet 91b6ef48e1b94812ac545e99dd49ba4b
I20260808 10:45:06.241003 12446 tablet_service.cc:1514] Processing DeleteTablet for tablet 91b6ef48e1b94812ac545e99dd49ba4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:06 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:59246
I20260808 10:45:06.241008 12209 tablet_service.cc:1514] Processing DeleteTablet for tablet 91b6ef48e1b94812ac545e99dd49ba4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:06 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:39502
I20260808 10:45:06.241200 12848 tablet_replica.cc:330] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae: stopping tablet replica
I20260808 10:45:06.241231 12851 tablet_replica.cc:330] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1: stopping tablet replica
I20260808 10:45:06.241355 12848 raft_consensus.cc:2229] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:45:06.241344 12670 tablet_service.cc:1514] Processing DeleteTablet for tablet 91b6ef48e1b94812ac545e99dd49ba4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:06 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:59726
I20260808 10:45:06.241397 12851 raft_consensus.cc:2229] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:45:06.241497 12848 raft_consensus.cc:2258] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:06.241513 12850 tablet_replica.cc:330] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650: stopping tablet replica
I20260808 10:45:06.241560 12851 raft_consensus.cc:2258] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:06.241629 12850 raft_consensus.cc:2229] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:45:06.241837 12850 raft_consensus.cc:2258] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:06.242259 12848 ts_tablet_manager.cc:1905] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:06.242520 12850 ts_tablet_manager.cc:1905] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:06.243041 12851 ts_tablet_manager.cc:1905] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:06.244578 12848 ts_tablet_manager.cc:1918] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:45:06.244635 12848 log.cc:1196] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/91b6ef48e1b94812ac545e99dd49ba4b
I20260808 10:45:06.244839 12848 ts_tablet_manager.cc:1939] T 91b6ef48e1b94812ac545e99dd49ba4b P 6710af7e2a094b979525619766dfb2ae: Deleting consensus metadata
I20260808 10:45:06.245008 12850 ts_tablet_manager.cc:1918] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:45:06.245074 12850 log.cc:1196] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/91b6ef48e1b94812ac545e99dd49ba4b
I20260808 10:45:06.245344 12850 ts_tablet_manager.cc:1939] T 91b6ef48e1b94812ac545e99dd49ba4b P b2340a1811ce46fc857fd0ba432de650: Deleting consensus metadata
I20260808 10:45:06.245476 11975 catalog_manager.cc:4841] TS 6710af7e2a094b979525619766dfb2ae (127.31.237.130:37139): tablet 91b6ef48e1b94812ac545e99dd49ba4b (table test-table [id=e46ef80e7d8d4472b58a15e98d151b07]) successfully deleted
I20260808 10:45:06.245596 12851 ts_tablet_manager.cc:1918] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:45:06.245664 12851 log.cc:1196] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-0/wal/wals/91b6ef48e1b94812ac545e99dd49ba4b
I20260808 10:45:06.245663 11996 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:45:06.245913 12851 ts_tablet_manager.cc:1939] T 91b6ef48e1b94812ac545e99dd49ba4b P 1c76622db89e413a8b7f4fd0d9fbc0b1: Deleting consensus metadata
W20260808 10:45:06.246075 11996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:45:06.246091 11971 catalog_manager.cc:4841] TS b2340a1811ce46fc857fd0ba432de650 (127.31.237.131:39527): tablet 91b6ef48e1b94812ac545e99dd49ba4b (table test-table [id=e46ef80e7d8d4472b58a15e98d151b07]) successfully deleted
I20260808 10:45:06.246680 11974 catalog_manager.cc:4841] TS 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129:43229): tablet 91b6ef48e1b94812ac545e99dd49ba4b (table test-table [id=e46ef80e7d8d4472b58a15e98d151b07]) successfully deleted
I20260808 10:45:06.250826 12209 tablet_service.cc:1467] Processing CreateTablet for tablet dea6a6e11f554153b3156cd01e16d065 (DEFAULT_TABLE table=test-table [id=95f90d9300314605b803c8b7c5b629ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:06.250895 12446 tablet_service.cc:1467] Processing CreateTablet for tablet dea6a6e11f554153b3156cd01e16d065 (DEFAULT_TABLE table=test-table [id=95f90d9300314605b803c8b7c5b629ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:06.251122 12209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea6a6e11f554153b3156cd01e16d065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:06.251185 12446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea6a6e11f554153b3156cd01e16d065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:06.253090 12812 tablet_bootstrap.cc:492] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1: Bootstrap starting.
I20260808 10:45:06.253178 12813 tablet_bootstrap.cc:492] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae: Bootstrap starting.
I20260808 10:45:06.253458 12670 tablet_service.cc:1467] Processing CreateTablet for tablet dea6a6e11f554153b3156cd01e16d065 (DEFAULT_TABLE table=test-table [id=95f90d9300314605b803c8b7c5b629ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:06.253666 12670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea6a6e11f554153b3156cd01e16d065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:06.253948 12812 tablet_bootstrap.cc:654] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:06.253948 12813 tablet_bootstrap.cc:654] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:06.255220 12813 tablet_bootstrap.cc:492] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae: No bootstrap required, opened a new log
I20260808 10:45:06.255312 12813 ts_tablet_manager.cc:1397] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:45:06.255688 12813 raft_consensus.cc:348] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:06.255826 12813 raft_consensus.cc:374] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:06.255864 12813 raft_consensus.cc:729] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6710af7e2a094b979525619766dfb2ae, State: Initialized, Role: FOLLOWER
I20260808 10:45:06.255911 12815 tablet_bootstrap.cc:492] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650: Bootstrap starting.
I20260808 10:45:06.255988 12813 consensus_queue.cc:260] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:06.256261 12813 ts_tablet_manager.cc:1428] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:45:06.257011 12815 tablet_bootstrap.cc:654] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:06.257143 12812 tablet_bootstrap.cc:492] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1: No bootstrap required, opened a new log
I20260808 10:45:06.257236 12812 ts_tablet_manager.cc:1397] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 10:45:06.257716 12812 raft_consensus.cc:348] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:06.257845 12812 raft_consensus.cc:374] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:06.257887 12812 raft_consensus.cc:729] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c76622db89e413a8b7f4fd0d9fbc0b1, State: Initialized, Role: FOLLOWER
I20260808 10:45:06.258042 12812 consensus_queue.cc:260] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:06.258322 12812 ts_tablet_manager.cc:1428] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:45:06.258435 12815 tablet_bootstrap.cc:492] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650: No bootstrap required, opened a new log
I20260808 10:45:06.258539 12815 ts_tablet_manager.cc:1397] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 10:45:06.258991 12815 raft_consensus.cc:348] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:06.259105 12815 raft_consensus.cc:374] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:06.259146 12815 raft_consensus.cc:729] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2340a1811ce46fc857fd0ba432de650, State: Initialized, Role: FOLLOWER
I20260808 10:45:06.259287 12815 consensus_queue.cc:260] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:06.259568 12815 ts_tablet_manager.cc:1428] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:45:06.416990 12858 raft_consensus.cc:482] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:45:06.417143 12858 raft_consensus.cc:504] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:06.417636 12858 leader_election.cc:288] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129:43229), 6710af7e2a094b979525619766dfb2ae (127.31.237.130:37139)
I20260808 10:45:06.418233 12487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea6a6e11f554153b3156cd01e16d065" candidate_uuid: "b2340a1811ce46fc857fd0ba432de650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6710af7e2a094b979525619766dfb2ae" is_pre_election: true
I20260808 10:45:06.418378 12487 raft_consensus.cc:2454] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2340a1811ce46fc857fd0ba432de650 in term 0.
I20260808 10:45:06.418320 12242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea6a6e11f554153b3156cd01e16d065" candidate_uuid: "b2340a1811ce46fc857fd0ba432de650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" is_pre_election: true
I20260808 10:45:06.418560 12242 raft_consensus.cc:2454] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2340a1811ce46fc857fd0ba432de650 in term 0.
I20260808 10:45:06.418826 12598 leader_election.cc:302] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6710af7e2a094b979525619766dfb2ae, b2340a1811ce46fc857fd0ba432de650; no voters:
I20260808 10:45:06.419037 12858 raft_consensus.cc:2781] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:45:06.419085 12858 raft_consensus.cc:482] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:45:06.419121 12858 raft_consensus.cc:3037] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:06.420384 12858 raft_consensus.cc:504] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:06.420745 12858 leader_election.cc:288] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [CANDIDATE]: Term 1 election: Requested vote from peers 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129:43229), 6710af7e2a094b979525619766dfb2ae (127.31.237.130:37139)
I20260808 10:45:06.421113 12487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea6a6e11f554153b3156cd01e16d065" candidate_uuid: "b2340a1811ce46fc857fd0ba432de650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6710af7e2a094b979525619766dfb2ae"
I20260808 10:45:06.421250 12487 raft_consensus.cc:3037] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:06.421190 12242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea6a6e11f554153b3156cd01e16d065" candidate_uuid: "b2340a1811ce46fc857fd0ba432de650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1"
I20260808 10:45:06.421346 12242 raft_consensus.cc:3037] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:06.422354 12487 raft_consensus.cc:2454] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2340a1811ce46fc857fd0ba432de650 in term 1.
I20260808 10:45:06.422384 12242 raft_consensus.cc:2454] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2340a1811ce46fc857fd0ba432de650 in term 1.
I20260808 10:45:06.422665 12598 leader_election.cc:302] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6710af7e2a094b979525619766dfb2ae, b2340a1811ce46fc857fd0ba432de650; no voters:
I20260808 10:45:06.422835 12858 raft_consensus.cc:2781] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:45:06.422964 12858 raft_consensus.cc:686] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [term 1 LEADER]: Becoming Leader. State: Replica: b2340a1811ce46fc857fd0ba432de650, State: Running, Role: LEADER
I20260808 10:45:06.423108 12858 consensus_queue.cc:237] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } }
I20260808 10:45:06.424993 11996 catalog_manager.cc:5495] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2340a1811ce46fc857fd0ba432de650 (127.31.237.131). New cstate: current_term: 1 leader_uuid: "b2340a1811ce46fc857fd0ba432de650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2340a1811ce46fc857fd0ba432de650" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39527 } health_report { overall_health: HEALTHY } } }
I20260808 10:45:06.460630 12487 raft_consensus.cc:1261] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Refusing update from remote peer b2340a1811ce46fc857fd0ba432de650: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:06.460783 12242 raft_consensus.cc:1261] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Refusing update from remote peer b2340a1811ce46fc857fd0ba432de650: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:06.461078 12824 consensus_queue.cc:1035] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6710af7e2a094b979525619766dfb2ae" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 37139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:06.461243 12858 consensus_queue.cc:1035] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c76622db89e413a8b7f4fd0d9fbc0b1" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:06.493575 11996 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:45:06.493733 11996 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:45:06.496161 11996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 431640e365114e1aaf692a07609e9ca0: Sending DeleteTablet for 3 replicas of tablet dea6a6e11f554153b3156cd01e16d065
I20260808 10:45:06.496837 12446 tablet_service.cc:1514] Processing DeleteTablet for tablet dea6a6e11f554153b3156cd01e16d065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:06 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:59246
I20260808 10:45:06.496884 12670 tablet_service.cc:1514] Processing DeleteTablet for tablet dea6a6e11f554153b3156cd01e16d065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:06 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:59726
I20260808 10:45:06.496902 12209 tablet_service.cc:1514] Processing DeleteTablet for tablet dea6a6e11f554153b3156cd01e16d065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:06 UTC) from {username='oryx', principal='oryx/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:39502
I20260808 10:45:06.497040 12848 tablet_replica.cc:330] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae: stopping tablet replica
I20260808 10:45:06.497068 12850 tablet_replica.cc:330] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650: stopping tablet replica
I20260808 10:45:06.497076 12851 tablet_replica.cc:330] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1: stopping tablet replica
I20260808 10:45:06.497174 12850 raft_consensus.cc:2229] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:45:06.497170 12851 raft_consensus.cc:2229] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:45:06.497167 12848 raft_consensus.cc:2229] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:45:06.497275 12848 raft_consensus.cc:2258] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:06.497280 12851 raft_consensus.cc:2258] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:06.497385 12850 raft_consensus.cc:2258] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:06.498111 12850 ts_tablet_manager.cc:1905] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:06.498118 12848 ts_tablet_manager.cc:1905] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:06.498324 12851 ts_tablet_manager.cc:1905] T dea6a6e11f554153b3156cd01e16d065 P 1c76622db89e413a8b7f4fd0d9fbc0b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:06.498895 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 12113
I20260808 10:45:06.500519 12850 ts_tablet_manager.cc:1918] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:45:06.500576 12850 log.cc:1196] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/dea6a6e11f554153b3156cd01e16d065
I20260808 10:45:06.500787 12850 ts_tablet_manager.cc:1939] T dea6a6e11f554153b3156cd01e16d065 P b2340a1811ce46fc857fd0ba432de650: Deleting consensus metadata
I20260808 10:45:06.501521 12848 ts_tablet_manager.cc:1918] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:45:06.501606 12848 log.cc:1196] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/dea6a6e11f554153b3156cd01e16d065
I20260808 10:45:06.501924 12848 ts_tablet_manager.cc:1939] T dea6a6e11f554153b3156cd01e16d065 P 6710af7e2a094b979525619766dfb2ae: Deleting consensus metadata
I20260808 10:45:06.504892 11971 catalog_manager.cc:4841] TS b2340a1811ce46fc857fd0ba432de650 (127.31.237.131:39527): tablet dea6a6e11f554153b3156cd01e16d065 (table test-table [id=95f90d9300314605b803c8b7c5b629ac]) successfully deleted
I20260808 10:45:06.504935 11975 catalog_manager.cc:4841] TS 6710af7e2a094b979525619766dfb2ae (127.31.237.130:37139): tablet dea6a6e11f554153b3156cd01e16d065 (table test-table [id=95f90d9300314605b803c8b7c5b629ac]) successfully deleted
W20260808 10:45:06.513752 11974 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:43229 (error 108)
I20260808 10:45:06.513998 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 12343
I20260808 10:45:06.527323 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:06.524912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:43229 (local address 127.0.0.1:39518)
0808 10:45:06.527143 (+ 2231us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:06.527219 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:43229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2128,"thread_start_us":768,"threads_started":1}
W20260808 10:45:06.527484 11974 catalog_manager.cc:4584] TS 1c76622db89e413a8b7f4fd0d9fbc0b1 (127.31.237.129:43229): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dea6a6e11f554153b3156cd01e16d065: Network error: Client connection negotiation failed: client connection to 127.31.237.129:43229: connect: Connection refused (error 111)
I20260808 10:45:06.534340 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 12573
I20260808 10:45:06.545356 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 11942
2026-08-08T10:45:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3776 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:45:06 dist-test-slave-twwt krb5kdc[12974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:45:06 dist-test-slave-twwt krb5kdc[12974](info): set up 2 sockets
Aug 08 10:45:06 dist-test-slave-twwt krb5kdc[12974](info): commencing operation
krb5kdc: starting...
W20260808 10:45:07.799254 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:45:07 dist-test-slave-twwt krb5kdc[12974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:45:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:45:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:45:07.927258 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:35359
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:34675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:35359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:45:08.045861 12997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:08.046164 12997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:08.046244 12997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:08.050220 12997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:45:08.050316 12997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:08.050355 12997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:08.050422 12997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:45:08.050462 12997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:45:08.055840 12997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:34675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:35359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:35359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:08.056910 12997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:08.057933 12997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:08.066664 13004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:08.066730 12997 server_base.cc:1034] running on GCE node
W20260808 10:45:08.066679 13002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:08.066689 13006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:08.067625 12997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:08.068698 12997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:08.069862 12997 hybrid_clock.cc:648] HybridClock initialized: now 1786185908069846 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T10:45:08Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1536 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:45:08 dist-test-slave-twwt krb5kdc[13017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:45:08 dist-test-slave-twwt krb5kdc[13017](info): set up 2 sockets
Aug 08 10:45:08 dist-test-slave-twwt krb5kdc[13017](info): commencing operation
krb5kdc: starting...
W20260808 10:45:09.333228 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:45:09 dist-test-slave-twwt krb5kdc[13017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:45:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:45:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:45:09.470055 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:33821
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:33821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:45:09.586535 13035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:09.586859 13035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:09.586941 13035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:09.590901 13035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:45:09.590991 13035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:09.591032 13035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:09.591094 13035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:45:09.591133 13035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:45:09.596391 13035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:33821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:33821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:09.597497 13035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:09.598533 13035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:09.607200 13049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:09.607172 13047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:09.607195 13042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:09.607602 13035 server_base.cc:1034] running on GCE node
I20260808 10:45:09.608162 13035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:09.609270 13035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:09.610456 13035 hybrid_clock.cc:648] HybridClock initialized: now 1786185909610421 us; error 44 us; skew 500 ppm
Aug 08 10:45:09 dist-test-slave-twwt krb5kdc[13017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185909, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:09.613387 13035 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:45:09.614239 13035 webserver.cc:460] Webserver started at http://127.31.237.190:33323/ using document root <none> and password file <none>
I20260808 10:45:09.614849 13035 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:09.614917 13035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:09.615090 13035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:09.617128 13035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "2674ad6852a44e1e926ba2646c42e645"
format_stamp: "Formatted at 2026-08-08 10:45:09 on dist-test-slave-twwt"
server_key: "db4988b268d29e9369087919e9853a35"
server_key_iv: "1810be64f07b9d665a8b9e3f4f1ef98a"
server_key_version: "encryptionkey@0"
I20260808 10:45:09.617659 13035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "2674ad6852a44e1e926ba2646c42e645"
format_stamp: "Formatted at 2026-08-08 10:45:09 on dist-test-slave-twwt"
server_key: "db4988b268d29e9369087919e9853a35"
server_key_iv: "1810be64f07b9d665a8b9e3f4f1ef98a"
server_key_version: "encryptionkey@0"
I20260808 10:45:09.621474 13035 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:45:09.623929 13059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:09.625278 13035 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:45:09.625444 13035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "2674ad6852a44e1e926ba2646c42e645"
format_stamp: "Formatted at 2026-08-08 10:45:09 on dist-test-slave-twwt"
server_key: "db4988b268d29e9369087919e9853a35"
server_key_iv: "1810be64f07b9d665a8b9e3f4f1ef98a"
server_key_version: "encryptionkey@0"
I20260808 10:45:09.625563 13035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:09.636628 13035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:09.639765 13035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:09.640009 13035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:09.647190 13035 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:33821
I20260808 10:45:09.647269 13140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:33821 every 8 connection(s)
I20260808 10:45:09.648202 13035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:45:09.651199 13141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:09.656896 13141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645: Bootstrap starting.
I20260808 10:45:09.657330 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 13035
I20260808 10:45:09.657474 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:45:09.657773 32694 external_mini_cluster.cc:1428] Setting key f163a29842f8b4b943225333c3af101f
I20260808 10:45:09.659670 13141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:09.660590 13141 log.cc:824] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:09.662894 13141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645: No bootstrap required, opened a new log
I20260808 10:45:09.667057 13141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2674ad6852a44e1e926ba2646c42e645" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33821 } }
Aug 08 10:45:09 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:45:09.667312 13141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:09.667440 13141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2674ad6852a44e1e926ba2646c42e645, State: Initialized, Role: FOLLOWER
I20260808 10:45:09.667977 13141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2674ad6852a44e1e926ba2646c42e645" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33821 } }
I20260808 10:45:09.668114 13141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:45:09.668198 13141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:45:09.668316 13141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:09.669399 13141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2674ad6852a44e1e926ba2646c42e645" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33821 } }
I20260808 10:45:09.669760 13141 leader_election.cc:302] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2674ad6852a44e1e926ba2646c42e645; no voters:
I20260808 10:45:09.670094 13141 leader_election.cc:288] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:45:09.670181 13149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:45:09.670449 13149 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 1 LEADER]: Becoming Leader. State: Replica: 2674ad6852a44e1e926ba2646c42e645, State: Running, Role: LEADER
I20260808 10:45:09.670846 13149 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2674ad6852a44e1e926ba2646c42e645" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33821 } }
I20260808 10:45:09.671667 13141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:45:09.672533 13151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2674ad6852a44e1e926ba2646c42e645. Latest consensus state: current_term: 1 leader_uuid: "2674ad6852a44e1e926ba2646c42e645" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2674ad6852a44e1e926ba2646c42e645" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33821 } } }
I20260808 10:45:09.672961 13151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:09.672858 13150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2674ad6852a44e1e926ba2646c42e645" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2674ad6852a44e1e926ba2646c42e645" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33821 } } }
I20260808 10:45:09.673295 13150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:09.673914 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:09.659225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55398 (local address 127.31.237.190:33821)
0808 10:45:09.659653 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:09.659681 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:09.659729 (+ 48us) server_negotiation.cc:407] Connection header received
0808 10:45:09.660605 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:09.660633 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:09.661064 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:09.661397 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:09.662100 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:09.663595 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:09.664434 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:09.665139 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:09.665309 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:09.667976 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:09.667981 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:09.667994 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:09.668019 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:09.669799 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:09.670399 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:09.670409 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:09.670418 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:09.670577 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:09.670927 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:09.670930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:09.670932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:09.671581 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:09.671820 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:09.672493 (+ 673us) server_negotiation.cc:299] Negotiation successful
0808 10:45:09.672853 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":96,"threads_started":1}
I20260808 10:45:09.674098 13160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:45:09.676994 13160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:45:09.682390 13160 catalog_manager.cc:1347] Generated new cluster ID: aa4185e82c1e464d80c3d7724ade6762
I20260808 10:45:09.682449 13160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:45:09.704027 13160 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:45:09.705276 13160 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:45:09.719184 13160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645: Generated new TSK 0
I20260808 10:45:09.719980 13160 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:45:09.762200 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33821
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:45:09.909092 13176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:09.909358 13176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:09.909401 13176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:09.913362 13176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:09.913414 13176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:09.913470 13176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:45:09.918541 13176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:09.919490 13176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:09.920454 13176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:09.930225 13187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:09.930181 13185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:09.930297 13184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:09.930339 13176 server_base.cc:1034] running on GCE node
I20260808 10:45:09.930907 13176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:09.931599 13176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:09.932796 13176 hybrid_clock.cc:648] HybridClock initialized: now 1786185909932779 us; error 32 us; skew 500 ppm
Aug 08 10:45:09 dist-test-slave-twwt krb5kdc[13017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185909, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:09.935621 13176 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:45:09.936491 13176 webserver.cc:460] Webserver started at http://127.31.237.129:36087/ using document root <none> and password file <none>
I20260808 10:45:09.937073 13176 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:09.937166 13176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:09.937414 13176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:09.939513 13176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "911e791344cd4947bdc664f5179e52f4"
format_stamp: "Formatted at 2026-08-08 10:45:09 on dist-test-slave-twwt"
server_key: "256adfb594a010b81f918ab1035339dc"
server_key_iv: "56aaff8e3332c4944ecc19aee3140963"
server_key_version: "encryptionkey@0"
I20260808 10:45:09.940090 13176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "911e791344cd4947bdc664f5179e52f4"
format_stamp: "Formatted at 2026-08-08 10:45:09 on dist-test-slave-twwt"
server_key: "256adfb594a010b81f918ab1035339dc"
server_key_iv: "56aaff8e3332c4944ecc19aee3140963"
server_key_version: "encryptionkey@0"
I20260808 10:45:09.943862 13176 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:45:09.946233 13197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:09.947489 13176 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:45:09.947630 13176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "911e791344cd4947bdc664f5179e52f4"
format_stamp: "Formatted at 2026-08-08 10:45:09 on dist-test-slave-twwt"
server_key: "256adfb594a010b81f918ab1035339dc"
server_key_iv: "56aaff8e3332c4944ecc19aee3140963"
server_key_version: "encryptionkey@0"
I20260808 10:45:09.947742 13176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:09.969516 13176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:09.973272 13176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:09.973490 13176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:09.974182 13176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:09.975221 13176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:09.975317 13176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:09.975391 13176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:09.975459 13176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:09.985003 13176 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:44729
I20260808 10:45:09.985025 13361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:44729 every 8 connection(s)
I20260808 10:45:09.985982 13176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:45:09.991329 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 13176
I20260808 10:45:09.991495 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:45:09.991870 32694 external_mini_cluster.cc:1428] Setting key 0f40f59fbe8a3a9235bba09b297913f6
Aug 08 10:45:09 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185909, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:45:10.000336 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:09.987852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:45727 (local address 127.31.237.190:33821)
0808 10:45:09.988022 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:09.988027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:09.988879 (+ 852us) server_negotiation.cc:407] Connection header received
0808 10:45:09.990146 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:09.990149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:09.990197 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:09.990272 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:09.991790 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:09.992365 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:09.993559 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:09.993806 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:09.993919 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:09.996667 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:09.996671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:09.996673 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:09.996696 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:09.998189 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:09.998763 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:09.998767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:09.998769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:09.998867 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:09.999230 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:09.999233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:09.999235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:09.999418 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:09.999525 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:10.000029 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 10:45:10.000161 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 10:45:10.001147 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:09.988265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:45727)
0808 10:45:09.988724 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:09.988767 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:09.989942 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:09.990442 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:09.990448 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:09.990912 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:09.991628 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:09.991650 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:09.992501 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:09.992505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:09.993427 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:09.993438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:09.993962 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:09.993965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:09.994139 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:09.994770 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:09.994792 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:09.996484 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:09.998325 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:09.998331 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:09.998346 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:09.998655 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:09.998967 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:09.998971 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:09.998974 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:09.999124 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:09.999525 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:09.999534 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:09.999786 (+ 252us) client_negotiation.cc:769] Sending connection context
0808 10:45:09.999969 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 10:45:10.000309 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:45:10.002457 13364 heartbeater.cc:347] Connected to a master server at 127.31.237.190:33821
I20260808 10:45:10.002746 13364 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:10.003329 13364 heartbeater.cc:511] Master 127.31.237.190:33821 requested a full tablet report, sending...
I20260808 10:45:10.005095 13085 ts_manager.cc:194] Registered new tserver with Master: 911e791344cd4947bdc664f5179e52f4 (127.31.237.129:44729)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
I20260808 10:45:10.006271 13085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:45727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:45:10.022864 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33821
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:45:10.145045 13371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:10.145308 13371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:10.145380 13371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:10.149431 13371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:10.149497 13371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:10.149588 13371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:45:10.155030 13371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:10.156059 13371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:10.157101 13371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:10.167281 13379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:10.167313 13380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:10.167281 13386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:10.167418 13371 server_base.cc:1034] running on GCE node
I20260808 10:45:10.168097 13371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:10.168779 13371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:10.169951 13371 hybrid_clock.cc:648] HybridClock initialized: now 1786185910169936 us; error 31 us; skew 500 ppm
Aug 08 10:45:10 dist-test-slave-twwt krb5kdc[13017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185910, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:10.173172 13371 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:45:10.174141 13371 webserver.cc:460] Webserver started at http://127.31.237.130:42835/ using document root <none> and password file <none>
I20260808 10:45:10.174908 13371 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:10.175000 13371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:10.175246 13371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:10.177379 13371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "30c938f4e3a5437880f63d82569d4a68"
format_stamp: "Formatted at 2026-08-08 10:45:10 on dist-test-slave-twwt"
server_key: "ff7fb6297178c3c25e8a175d058c7626"
server_key_iv: "b3fb75fa344bd48daadf0cc66ad7e2c9"
server_key_version: "encryptionkey@0"
I20260808 10:45:10.177980 13371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "30c938f4e3a5437880f63d82569d4a68"
format_stamp: "Formatted at 2026-08-08 10:45:10 on dist-test-slave-twwt"
server_key: "ff7fb6297178c3c25e8a175d058c7626"
server_key_iv: "b3fb75fa344bd48daadf0cc66ad7e2c9"
server_key_version: "encryptionkey@0"
I20260808 10:45:10.182044 13371 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:45:10.184901 13396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:10.186213 13371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 10:45:10.186317 13371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "30c938f4e3a5437880f63d82569d4a68"
format_stamp: "Formatted at 2026-08-08 10:45:10 on dist-test-slave-twwt"
server_key: "ff7fb6297178c3c25e8a175d058c7626"
server_key_iv: "b3fb75fa344bd48daadf0cc66ad7e2c9"
server_key_version: "encryptionkey@0"
I20260808 10:45:10.186472 13371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:10.200908 13371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:10.204133 13371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:10.204327 13371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:10.204933 13371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:10.205742 13371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:10.205788 13371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:10.205859 13371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:10.205902 13371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:10.214932 13371 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:45035
I20260808 10:45:10.214964 13571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:45035 every 8 connection(s)
I20260808 10:45:10.215830 13371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:45:10.220927 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 13371
I20260808 10:45:10.221078 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:45:10.221360 32694 external_mini_cluster.cc:1428] Setting key d5559c035b52e9e874a03d772fa65c0c
Aug 08 10:45:10 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185910, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:45:10.230751 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:10.217713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:34175 (local address 127.31.237.190:33821)
0808 10:45:10.217861 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:10.217865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:10.219041 (+ 1176us) server_negotiation.cc:407] Connection header received
0808 10:45:10.219923 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:10.219926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:10.219971 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:10.220044 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:10.221292 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:10.221972 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:10.223327 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:10.223549 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:10.223660 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:10.226355 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:10.226361 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:10.226367 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:10.226403 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:10.228238 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:10.228862 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:10.228867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:10.228868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:10.228977 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:10.229383 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:10.229389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:10.229393 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:10.229703 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:10.229844 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:10.230356 (+ 512us) server_negotiation.cc:299] Negotiation successful
0808 10:45:10.230493 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:45:10.231508 13577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:10.218086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:34175)
0808 10:45:10.218867 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:10.218911 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:10.219717 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:10.220214 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:10.220221 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:10.220591 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:10.221114 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:10.221154 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:10.222099 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:10.222103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:10.223192 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:10.223204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:10.223703 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:10.223707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:10.223886 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:10.224493 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:10.224510 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:10.226202 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:10.228384 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:10.228392 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:10.228409 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:10.228775 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:10.229106 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:10.229109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:10.229111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:10.229272 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:10.229830 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:10.229840 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:10.230100 (+ 260us) client_negotiation.cc:769] Sending connection context
0808 10:45:10.230266 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 10:45:10.230532 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":105,"threads_started":1}
I20260808 10:45:10.232749 13574 heartbeater.cc:347] Connected to a master server at 127.31.237.190:33821
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:45:10.233009 13574 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:10.233534 13574 heartbeater.cc:511] Master 127.31.237.190:33821 requested a full tablet report, sending...
I20260808 10:45:10.234714 13085 ts_manager.cc:194] Registered new tserver with Master: 30c938f4e3a5437880f63d82569d4a68 (127.31.237.130:45035)
I20260808 10:45:10.235360 13085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:34175
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:45:10.253212 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33821
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:45:10.370501 13583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:10.370826 13583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:10.370909 13583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:10.375209 13583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:10.375321 13583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:10.375422 13583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:45:10.380824 13583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:10.381950 13583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:10.383100 13583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:10.392926 13592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:10.393055 13593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:10.392926 13598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:10.393205 13583 server_base.cc:1034] running on GCE node
I20260808 10:45:10.393687 13583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:10.394366 13583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:10.395552 13583 hybrid_clock.cc:648] HybridClock initialized: now 1786185910395515 us; error 48 us; skew 500 ppm
Aug 08 10:45:10 dist-test-slave-twwt krb5kdc[13017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185910, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:10.398316 13583 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:45:10.399238 13583 webserver.cc:460] Webserver started at http://127.31.237.131:33289/ using document root <none> and password file <none>
I20260808 10:45:10.399806 13583 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:10.399921 13583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:10.400161 13583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:10.402168 13583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "b8e037e63a3d407fae5d3df1d01aec42"
format_stamp: "Formatted at 2026-08-08 10:45:10 on dist-test-slave-twwt"
server_key: "b90427295b7a38d56200acc923455355"
server_key_iv: "92b00350f7dad1639340f7026abdde0e"
server_key_version: "encryptionkey@0"
I20260808 10:45:10.402789 13583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "b8e037e63a3d407fae5d3df1d01aec42"
format_stamp: "Formatted at 2026-08-08 10:45:10 on dist-test-slave-twwt"
server_key: "b90427295b7a38d56200acc923455355"
server_key_iv: "92b00350f7dad1639340f7026abdde0e"
server_key_version: "encryptionkey@0"
I20260808 10:45:10.406517 13583 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:45:10.409060 13610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:10.410378 13583 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:45:10.410537 13583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "b8e037e63a3d407fae5d3df1d01aec42"
format_stamp: "Formatted at 2026-08-08 10:45:10 on dist-test-slave-twwt"
server_key: "b90427295b7a38d56200acc923455355"
server_key_iv: "92b00350f7dad1639340f7026abdde0e"
server_key_version: "encryptionkey@0"
I20260808 10:45:10.410688 13583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:10.427094 13583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:10.430789 13583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:10.431001 13583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:10.431670 13583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:10.432593 13583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:10.432668 13583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:10.432756 13583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:10.432811 13583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:10.441907 13583 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:35393
I20260808 10:45:10.441926 13795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:35393 every 8 connection(s)
I20260808 10:45:10.442924 13583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:45:10.451035 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 13583
I20260808 10:45:10.451205 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:45:10.451577 32694 external_mini_cluster.cc:1428] Setting key 932e0d03715012ff482a86e3096f797f
Aug 08 10:45:10 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185910, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:45:10.456857 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:10.444798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:50799 (local address 127.31.237.190:33821)
0808 10:45:10.444947 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:10.444950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:10.446062 (+ 1112us) server_negotiation.cc:407] Connection header received
0808 10:45:10.447042 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:10.447045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:10.447092 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:10.447166 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:10.448350 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:10.448876 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:10.450039 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:10.450245 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:10.450319 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:10.453484 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:10.453492 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:10.453494 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:10.453520 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:10.455022 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:10.455518 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:10.455520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:10.455521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:10.455597 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:10.455914 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:10.455916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:10.455918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:10.456094 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:10.456169 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:10.456643 (+ 474us) server_negotiation.cc:299] Negotiation successful
0808 10:45:10.456739 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:45:10.457753 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:10.445172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:50799)
0808 10:45:10.445891 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:10.445957 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:10.446840 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:10.447336 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:10.447343 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:10.447707 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:10.448206 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:10.448220 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:10.449014 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:10.449017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:10.449928 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:10.449938 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:10.450387 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:10.450389 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:10.450529 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:10.451262 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:10.451286 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:10.453296 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:10.455127 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:10.455133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:10.455146 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:10.455431 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:10.455674 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:10.455676 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:10.455678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:10.455822 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:10.456179 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:10.456185 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:10.456436 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 10:45:10.456597 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 10:45:10.456854 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":119,"threads_started":1}
I20260808 10:45:10.459008 13796 heartbeater.cc:347] Connected to a master server at 127.31.237.190:33821
I20260808 10:45:10.459260 13796 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:10.459784 13796 heartbeater.cc:511] Master 127.31.237.190:33821 requested a full tablet report, sending...
I20260808 10:45:10.460850 13085 ts_manager.cc:194] Registered new tserver with Master: b8e037e63a3d407fae5d3df1d01aec42 (127.31.237.131:35393)
I20260808 10:45:10.461436 13085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:50799
I20260808 10:45:10.465597 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.129.keytab.
Entry for principal oryx/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.129.keytab.
WARNING: no policy specified for oryx/127.31.237.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.237.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.130.keytab.
Entry for principal oryx/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.130.keytab.
WARNING: no policy specified for oryx/127.31.237.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.237.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.131.keytab.
Entry for principal oryx/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.131.keytab.
I20260808 10:45:10.545302 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 13176
I20260808 10:45:10.559468 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 13371
I20260808 10:45:10.574568 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 13583
I20260808 10:45:10.605069 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 13035
I20260808 10:45:10.618076 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:33821
--webserver_interface=127.31.237.190
--webserver_port=33323
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:33821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:45:10.733520 13812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:10.733809 13812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:10.733851 13812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:10.737910 13812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:45:10.737983 13812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:10.738003 13812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:10.738020 13812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:45:10.738036 13812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:45:10.743595 13812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:33821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:33821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=33323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:10.744807 13812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:10.745909 13812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:45:10.755031 13812 server_base.cc:1034] running on GCE node
W20260808 10:45:10.754990 13817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:10.755016 13819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:10.754990 13821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:10.755771 13812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:10.756969 13812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:10.758147 13812 hybrid_clock.cc:648] HybridClock initialized: now 1786185910758124 us; error 36 us; skew 500 ppm
Aug 08 10:45:10 dist-test-slave-twwt krb5kdc[13017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185910, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:10.761246 13812 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:45:10.762261 13812 webserver.cc:460] Webserver started at http://127.31.237.190:33323/ using document root <none> and password file <none>
I20260808 10:45:10.762946 13812 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:10.763032 13812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:10.767340 13812 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 10:45:10.769706 13831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:10.771227 13812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:45:10.771387 13812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "2674ad6852a44e1e926ba2646c42e645"
format_stamp: "Formatted at 2026-08-08 10:45:09 on dist-test-slave-twwt"
server_key: "db4988b268d29e9369087919e9853a35"
server_key_iv: "1810be64f07b9d665a8b9e3f4f1ef98a"
server_key_version: "encryptionkey@0"
I20260808 10:45:10.771992 13812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:10.789685 13812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:10.793269 13812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:10.793500 13812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:10.801316 13812 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:33821
I20260808 10:45:10.801348 13906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:33821 every 8 connection(s)
I20260808 10:45:10.802456 13812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:45:10.805400 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 13812
I20260808 10:45:10.805935 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:44729
--local_ip_for_outbound_sockets=127.31.237.129
--tserver_master_addrs=127.31.237.190:33821
--webserver_port=36087
--webserver_interface=127.31.237.129
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.237.129
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:45:10.808099 13909 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:45:10.809937 13909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645: Bootstrap starting.
I20260808 10:45:10.827414 13909 log.cc:824] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:10.834882 13909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:45:10.835302 13909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645: Bootstrap complete.
I20260808 10:45:10.839797 13909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2674ad6852a44e1e926ba2646c42e645" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33821 } }
I20260808 10:45:10.840268 13909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2674ad6852a44e1e926ba2646c42e645, State: Initialized, Role: FOLLOWER
I20260808 10:45:10.840866 13909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2674ad6852a44e1e926ba2646c42e645" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33821 } }
I20260808 10:45:10.841027 13909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:45:10.841127 13909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:45:10.841257 13909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:45:10.843472 13909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2674ad6852a44e1e926ba2646c42e645" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33821 } }
I20260808 10:45:10.843843 13909 leader_election.cc:302] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2674ad6852a44e1e926ba2646c42e645; no voters:
I20260808 10:45:10.844200 13909 leader_election.cc:288] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:45:10.844364 13918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:45:10.844622 13918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [term 2 LEADER]: Becoming Leader. State: Replica: 2674ad6852a44e1e926ba2646c42e645, State: Running, Role: LEADER
I20260808 10:45:10.844959 13918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2674ad6852a44e1e926ba2646c42e645" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33821 } }
I20260808 10:45:10.845506 13909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:45:10.846645 13919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2674ad6852a44e1e926ba2646c42e645" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2674ad6852a44e1e926ba2646c42e645" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33821 } } }
I20260808 10:45:10.846781 13919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:10.846760 13920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2674ad6852a44e1e926ba2646c42e645. Latest consensus state: current_term: 2 leader_uuid: "2674ad6852a44e1e926ba2646c42e645" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2674ad6852a44e1e926ba2646c42e645" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33821 } } }
I20260808 10:45:10.847158 13920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645 [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:10.848131 13931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:45:10.851524 13931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:45:10.852290 13931 catalog_manager.cc:1259] Loaded cluster ID: aa4185e82c1e464d80c3d7724ade6762
I20260808 10:45:10.852353 13931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:45:10.854813 13931 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:45:10.855515 13931 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2674ad6852a44e1e926ba2646c42e645: Loaded TSK: 0
I20260808 10:45:10.856801 13931 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:45:10.948300 13914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:10.948596 13914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:10.948668 13914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:10.953058 13914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:10.953150 13914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:10.953271 13914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:45:10.958577 13914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.129.keytab
--principal=oryx/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:44729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=36087
--tserver_master_addrs=127.31.237.190:33821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:10.959692 13914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:10.960748 13914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:10.970702 13957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:10.970713 13951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:10.970798 13955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:10.972035 13914 server_base.cc:1034] running on GCE node
I20260808 10:45:10.972514 13914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:10.973280 13914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:10.974490 13914 hybrid_clock.cc:648] HybridClock initialized: now 1786185910974446 us; error 64 us; skew 500 ppm
Aug 08 10:45:10 dist-test-slave-twwt krb5kdc[13017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185910, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:10.977932 13914 init.cc:377] Logged in from keytab as oryx/127.31.237.129@KRBTEST.COM (short username oryx)
I20260808 10:45:10.979132 13914 webserver.cc:460] Webserver started at http://127.31.237.129:36087/ using document root <none> and password file <none>
I20260808 10:45:10.979751 13914 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:10.979840 13914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:10.984207 13914 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:45:10.986671 13969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:10.988054 13914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:45:10.988185 13914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "911e791344cd4947bdc664f5179e52f4"
format_stamp: "Formatted at 2026-08-08 10:45:09 on dist-test-slave-twwt"
server_key: "256adfb594a010b81f918ab1035339dc"
server_key_iv: "56aaff8e3332c4944ecc19aee3140963"
server_key_version: "encryptionkey@0"
I20260808 10:45:10.988770 13914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:11.010828 13914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:11.014478 13914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:11.014715 13914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:11.015430 13914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:11.016391 13914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:11.016443 13914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:11.016525 13914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:11.016556 13914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:11.026760 13914 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:44729
I20260808 10:45:11.026790 14147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:44729 every 8 connection(s)
I20260808 10:45:11.027895 13914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:45:11.034863 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 13914
I20260808 10:45:11.035401 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:45035
--local_ip_for_outbound_sockets=127.31.237.130
--tserver_master_addrs=127.31.237.190:33821
--webserver_port=42835
--webserver_interface=127.31.237.130
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.237.130
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:11.042383 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:11.030158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:46095)
0808 10:45:11.030954 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:11.031007 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:11.031810 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:11.032978 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:11.032984 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:11.033339 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:11.033816 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.033831 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.035335 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.035339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.036794 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.036810 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.037631 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.037637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.037862 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.038582 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:11.038643 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:11.041547 (+ 2904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":171,"threads_started":1}
W20260808 10:45:11.042974 14148 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:33821 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.237.190:33821: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
W20260808 10:45:11.043502 14151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:11.030075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:46095 (local address 127.31.237.190:33821)
0808 10:45:11.030528 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:11.030544 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:11.031154 (+ 610us) server_negotiation.cc:407] Connection header received
0808 10:45:11.032234 (+ 1080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:11.032246 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:11.032560 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:11.032864 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:11.034002 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.035191 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.036944 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.037447 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.037588 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.041885 (+ 4297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:46095: BlockingRecv error: recv got EOF from 127.31.237.129:46095 (error 108)
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":119,"threads_started":1}
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:11.050037 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:11.043514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:45671)
0808 10:45:11.043731 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:11.043749 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:11.043840 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:11.044246 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:11.044251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:11.044454 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:11.044605 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.044615 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.045582 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.045587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.046725 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.046740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.047188 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.047193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.047320 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.047900 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:11.047922 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:11.049943 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125}
W20260808 10:45:11.050343 14151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:11.043620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:45671 (local address 127.31.237.190:33821)
0808 10:45:11.043990 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:11.043994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:11.044009 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:45:11.044051 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:11.044054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:11.044108 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:11.044193 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:11.044738 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.045452 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.046851 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.047065 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.047146 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.050109 (+ 2963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:45671: BlockingRecv error: recv got EOF from 127.31.237.129:45671 (error 108)
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":115,"threads_started":1}
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:11.055575 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:11.050598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:58239)
0808 10:45:11.050805 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:11.050820 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:11.050928 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:11.051186 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:11.051189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:11.051348 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:11.051416 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.051421 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.052154 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.052158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.052679 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.052686 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.053136 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.053140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.053221 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.053702 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:11.053714 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:11.055437 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118}
W20260808 10:45:11.055665 14151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:11.050690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:58239 (local address 127.31.237.190:33821)
0808 10:45:11.050824 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:11.050828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:11.050900 (+ 72us) server_negotiation.cc:407] Connection header received
0808 10:45:11.051035 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:11.051037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:11.051087 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:11.051171 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:11.051528 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.052038 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.052815 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.053027 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.053134 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.055553 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:58239: BlockingRecv error: recv got EOF from 127.31.237.129:58239 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 10:45:11.055786 14148 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:45:11.157049 14153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:11.157307 14153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:11.157351 14153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:11.161332 14153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:11.161396 14153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:11.161453 14153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:45:11.166934 14153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.130.keytab
--principal=oryx/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:45035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=42835
--tserver_master_addrs=127.31.237.190:33821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:11.168110 14153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:11.169157 14153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:11.179484 14165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:11.179482 14168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:11.179524 14166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:11.179813 14153 server_base.cc:1034] running on GCE node
I20260808 10:45:11.180305 14153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:11.180965 14153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:11.182157 14153 hybrid_clock.cc:648] HybridClock initialized: now 1786185911182137 us; error 29 us; skew 500 ppm
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185911, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:11.185186 14153 init.cc:377] Logged in from keytab as oryx/127.31.237.130@KRBTEST.COM (short username oryx)
I20260808 10:45:11.186188 14153 webserver.cc:460] Webserver started at http://127.31.237.130:42835/ using document root <none> and password file <none>
I20260808 10:45:11.186897 14153 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:11.187016 14153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:11.190999 14153 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260808 10:45:11.193305 14179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:11.194587 14153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:45:11.194780 14153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "30c938f4e3a5437880f63d82569d4a68"
format_stamp: "Formatted at 2026-08-08 10:45:10 on dist-test-slave-twwt"
server_key: "ff7fb6297178c3c25e8a175d058c7626"
server_key_iv: "b3fb75fa344bd48daadf0cc66ad7e2c9"
server_key_version: "encryptionkey@0"
I20260808 10:45:11.195375 14153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:11.216763 14153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:11.220575 14153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:11.220803 14153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:11.221457 14153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:11.222345 14153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:11.222432 14153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:11.222527 14153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:11.222579 14153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:11.232031 14153 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:45035
I20260808 10:45:11.232053 14339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:45035 every 8 connection(s)
I20260808 10:45:11.233119 14153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:45:11.234045 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 14153
I20260808 10:45:11.234562 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:35393
--local_ip_for_outbound_sockets=127.31.237.131
--tserver_master_addrs=127.31.237.190:33821
--webserver_port=33289
--webserver_interface=127.31.237.131
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.237.131
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:45:11.246357 14151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:11.235139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:60811 (local address 127.31.237.190:33821)
0808 10:45:11.235286 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:11.235289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:11.236678 (+ 1389us) server_negotiation.cc:407] Connection header received
0808 10:45:11.238010 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:11.238012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:11.238065 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:11.238175 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:11.239547 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.240494 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.241713 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.242145 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.242281 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.246167 (+ 3886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:60811: BlockingRecv error: recv got EOF from 127.31.237.130:60811 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:45:11.247206 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:11.235596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:60811)
0808 10:45:11.236478 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:11.236545 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:11.237745 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:11.238395 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:11.238402 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:11.238858 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:11.239376 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.239390 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.240654 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.240658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.241574 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.241585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.242302 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.242304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.242461 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.243187 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:11.243204 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:11.245963 (+ 2759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":159,"threads_started":1}
W20260808 10:45:11.247906 14341 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:33821 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.237.190:33821: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:11.254127 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:11.248447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:60169)
0808 10:45:11.248572 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:11.248587 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:11.248674 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:11.248928 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:11.248930 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:11.249068 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:11.249167 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.249172 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.250203 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.250205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.250841 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.250851 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.251457 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.251460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.251536 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.252069 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:11.252081 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:11.254021 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 10:45:11.254657 14151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:11.248524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:60169 (local address 127.31.237.190:33821)
0808 10:45:11.248646 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:11.248650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:11.248665 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:45:11.248748 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:11.248754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:11.248802 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:11.248893 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:11.249274 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.250089 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.250971 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.251320 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.251427 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.254202 (+ 2775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:60169: BlockingRecv error: recv got EOF from 127.31.237.130:60169 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:11.259907 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:11.254788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:33117)
0808 10:45:11.254964 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:11.254979 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:11.255085 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:11.255381 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:11.255384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:11.255547 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:11.255619 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.255624 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.256397 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.256400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.256911 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.256918 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.257388 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.257392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.257482 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.257979 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:11.257990 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:11.259789 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109}
W20260808 10:45:11.260090 14155 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:11.254838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:33117 (local address 127.31.237.190:33821)
0808 10:45:11.254968 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:11.254978 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:11.255050 (+ 72us) server_negotiation.cc:407] Connection header received
0808 10:45:11.255202 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:11.255205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:11.255258 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:11.255350 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:11.255732 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.256272 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.257038 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.257264 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.257366 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.259971 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:33117: BlockingRecv error: recv got EOF from 127.31.237.130:33117 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 10:45:11.260331 14341 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:45:11.356971 14346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:11.357242 14346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:11.357327 14346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:11.361486 14346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:11.361550 14346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:11.361655 14346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:45:11.367635 14346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/krb5kdc/oryx_127.31.237.131.keytab
--principal=oryx/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:35393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=33289
--tserver_master_addrs=127.31.237.190:33821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:11.368942 14346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:11.370033 14346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:11.380344 14354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:11.380358 14352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:11.380348 14358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:11.380589 14346 server_base.cc:1034] running on GCE node
I20260808 10:45:11.381105 14346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:11.381790 14346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:11.383050 14346 hybrid_clock.cc:648] HybridClock initialized: now 1786185911383028 us; error 30 us; skew 500 ppm
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185911, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:11.386130 14346 init.cc:377] Logged in from keytab as oryx/127.31.237.131@KRBTEST.COM (short username oryx)
I20260808 10:45:11.387228 14346 webserver.cc:460] Webserver started at http://127.31.237.131:33289/ using document root <none> and password file <none>
I20260808 10:45:11.387779 14346 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:11.387852 14346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:11.391698 14346 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:45:11.394078 14368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:11.395390 14346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:45:11.395529 14346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "b8e037e63a3d407fae5d3df1d01aec42"
format_stamp: "Formatted at 2026-08-08 10:45:10 on dist-test-slave-twwt"
server_key: "b90427295b7a38d56200acc923455355"
server_key_iv: "92b00350f7dad1639340f7026abdde0e"
server_key_version: "encryptionkey@0"
I20260808 10:45:11.396101 14346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:11.418967 14346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:11.422523 14346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:11.422770 14346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:11.423457 14346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:11.424292 14346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:11.424350 14346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:11.424394 14346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:11.424413 14346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:11.433917 14346 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:35393
I20260808 10:45:11.433943 14554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:35393 every 8 connection(s)
I20260808 10:45:11.435005 14346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:45:11.443588 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 14346
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:45:11.446627 14155 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:11.436995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:52647 (local address 127.31.237.190:33821)
0808 10:45:11.437172 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:11.437177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:11.438224 (+ 1047us) server_negotiation.cc:407] Connection header received
0808 10:45:11.439110 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:11.439113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:11.439185 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:11.439263 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:11.440583 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.441127 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.442246 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.442458 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.442533 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.446483 (+ 3950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:52647: BlockingRecv error: recv got EOF from 127.31.237.131:52647 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 10:45:11.447301 14557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:11.437431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:52647)
0808 10:45:11.438076 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:11.438118 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:11.438918 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:11.439493 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:11.439500 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:11.439927 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:11.440423 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.440445 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.441258 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.441261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.442140 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.442150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.442584 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.442587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.442757 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.443450 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:11.443471 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:11.446343 (+ 2872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":101,"threads_started":1}
W20260808 10:45:11.448082 14555 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:33821 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.237.190:33821: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:11.453693 14557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:11.448552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:34685)
0808 10:45:11.448678 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:11.448694 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:11.448778 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:11.449021 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:11.449024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:11.449201 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:11.449341 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.449351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.450038 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.450042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.450588 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.450596 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.451085 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.451088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.451182 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.451612 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:11.451623 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:11.453543 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 10:45:11.453917 14155 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:11.448605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:34685 (local address 127.31.237.190:33821)
0808 10:45:11.448713 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:11.448716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:11.448732 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:45:11.448870 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:11.448873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:11.448909 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:11.448973 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:11.449459 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.449927 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.450758 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.450974 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.451081 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.453798 (+ 2717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:34685: BlockingRecv error: recv got EOF from 127.31.237.131:34685 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:45:11.454253 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:11.444663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55400 (local address 127.31.237.190:33821)
0808 10:45:11.444793 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:11.444799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:11.444813 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:45:11.444952 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:11.444956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:11.445003 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:11.445109 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:11.445985 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.446747 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.447625 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.447876 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.447984 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.449769 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:11.449775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:11.449791 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:11.449817 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:11.451749 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:11.452278 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:11.452283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:11.452289 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:11.452411 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:11.452743 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:11.452750 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:11.452754 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:11.453208 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:11.453427 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:11.453861 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 10:45:11.454139 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:11 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:11.459939 14557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:11.454897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:35731)
0808 10:45:11.454993 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:11.455008 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:11.455085 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:11.455282 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:11.455283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:11.455475 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:11.455567 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.455574 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.456418 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.456422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.457108 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:11.457119 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.457595 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.457597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:11.457656 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.458042 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:11.458052 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:11.459779 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 10:45:11.459995 14151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:11.454956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:35731 (local address 127.31.237.190:33821)
0808 10:45:11.455083 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:11.455087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:11.455100 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:45:11.455138 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:11.455140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:11.455177 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:11.455268 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:11.455676 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.456189 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.457255 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:11.457498 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:11.457574 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:11.459890 (+ 2316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:35731: BlockingRecv error: recv got EOF from 127.31.237.131:35731 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 10:45:11.460292 14555 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 10:45:12 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:12 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:12.066488 14563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:12.056516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:53019)
0808 10:45:12.056788 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:12.056823 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:12.056981 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:12.057248 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:12.057250 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:12.057436 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:12.057560 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:12.057565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:12.058278 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:12.058280 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:12.058886 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:12.058895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:12.059297 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:12.059300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:12.059371 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:12.064464 (+ 5093us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:12.064477 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:12.066365 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":91,"threads_started":1}
W20260808 10:45:12.066730 14564 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:12.056592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:53019 (local address 127.31.237.190:33821)
0808 10:45:12.056866 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:12.056871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:12.056910 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:45:12.057081 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:12.057083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:12.057130 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:12.057205 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:12.057661 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:12.058183 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:12.058985 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:12.059200 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:12.059275 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:12.066532 (+ 7257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:53019: BlockingRecv error: recv got EOF from 127.31.237.129:53019 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":118,"threads_started":1}
Aug 08 10:45:12 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:12 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:12.266650 14567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:12.261068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:34315)
0808 10:45:12.261374 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:12.261388 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:12.261493 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:12.261729 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:12.261731 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:12.261973 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:12.262097 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:12.262103 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:12.262909 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:12.262912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:12.263478 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:12.263486 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:12.263906 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:12.263909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:12.263982 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:12.264574 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:12.264586 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:12.266454 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":95,"threads_started":1}
W20260808 10:45:12.266713 14564 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:12.261139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:34315 (local address 127.31.237.190:33821)
0808 10:45:12.261325 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:12.261330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:12.261460 (+ 130us) server_negotiation.cc:407] Connection header received
0808 10:45:12.261576 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:12.261579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:12.261626 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:12.261693 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:12.262208 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:12.262799 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:12.263586 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:12.263799 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:12.263876 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:12.266578 (+ 2702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:34315: BlockingRecv error: recv got EOF from 127.31.237.130:34315 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 10:45:12 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:12 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:12.466784 14568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:12.461061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:49593)
0808 10:45:12.461387 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:12.461401 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:12.461504 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:12.461738 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:12.461740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:12.461927 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:12.462052 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:12.462058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:12.462845 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:12.462848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:12.463495 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:12.463506 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:12.464071 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:12.464075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:12.464183 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:12.464671 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:12.464683 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:12.466585 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":114,"threads_started":1}
W20260808 10:45:12.466849 14564 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:12.461165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:49593 (local address 127.31.237.190:33821)
0808 10:45:12.461334 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:12.461344 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:12.461457 (+ 113us) server_negotiation.cc:407] Connection header received
0808 10:45:12.461574 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:12.461577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:12.461621 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:12.461698 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:12.462164 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:12.462701 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:12.463633 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:12.463948 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:12.464064 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:12.466739 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:49593: BlockingRecv error: recv got EOF from 127.31.237.131:49593 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 08 10:45:13 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:13 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:13.073190 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:13.067485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:57195)
0808 10:45:13.067783 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:13.067811 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:13.067934 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:13.068237 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:13.068240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:13.068467 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:13.068571 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:13.068577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:13.069380 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:13.069390 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:13.069941 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:13.069949 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:13.070425 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:13.070429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:13.070522 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:13.071084 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:13.071097 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:13.073004 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":97,"threads_started":1}
W20260808 10:45:13.073269 14570 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:13.067630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:57195 (local address 127.31.237.190:33821)
0808 10:45:13.067918 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:13.067923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:13.067939 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:45:13.068049 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:13.068052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:13.068106 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:13.068203 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:13.068702 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:13.069253 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:13.070069 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:13.070307 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:13.070417 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:13.073140 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:57195: BlockingRecv error: recv got EOF from 127.31.237.129:57195 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":81,"threads_started":1}
Aug 08 10:45:13 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:13 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:13.273311 14571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:13.267554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:36399)
0808 10:45:13.267857 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:13.267875 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:13.268003 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:13.268227 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:13.268229 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:13.268427 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:13.268524 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:13.268530 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:13.269241 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:13.269243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:13.269797 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:13.269805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:13.270351 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:13.270356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:13.270441 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:13.271004 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:13.271036 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:13.273179 (+ 2143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":130,"threads_started":1}
W20260808 10:45:13.273450 14570 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:13.267621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:36399 (local address 127.31.237.190:33821)
0808 10:45:13.267777 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:13.267781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:13.267958 (+ 177us) server_negotiation.cc:407] Connection header received
0808 10:45:13.268087 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:13.268090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:13.268138 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:13.268216 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:13.268624 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:13.269141 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:13.269933 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:13.270226 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:13.270309 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:13.273317 (+ 3008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:36399: BlockingRecv error: recv got EOF from 127.31.237.130:36399 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 10:45:13 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:13 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:13.473465 14573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:13.467621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:49865)
0808 10:45:13.467979 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:13.467997 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:13.468133 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:13.468437 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:13.468440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:13.468666 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:13.468760 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:13.468766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:13.469539 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:13.469544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:13.470292 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:13.470307 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:13.470954 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:13.470956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:13.471021 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:13.471499 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:13.471511 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:13.473340 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":112,"threads_started":1}
W20260808 10:45:13.473605 14570 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:13.467804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:49865 (local address 127.31.237.190:33821)
0808 10:45:13.467980 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:13.467986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:13.468090 (+ 104us) server_negotiation.cc:407] Connection header received
0808 10:45:13.468255 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:13.468261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:13.468320 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:13.468421 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:13.468883 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:13.469423 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:13.470455 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:13.470837 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:13.470919 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:13.473485 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:49865: BlockingRecv error: recv got EOF from 127.31.237.131:49865 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 08 10:45:14 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:14 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:14.079751 14574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:14.074063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:43839)
0808 10:45:14.074343 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:14.074357 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:14.074451 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:14.074732 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:14.074734 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:14.074986 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:14.075089 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:14.075094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:14.075932 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:14.075939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:14.076565 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:14.076574 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:14.077004 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:14.077007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:14.077069 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:14.077537 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:14.077548 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:14.079559 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":90,"threads_started":1}
W20260808 10:45:14.079815 14575 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:14.074141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:43839 (local address 127.31.237.190:33821)
0808 10:45:14.074389 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:14.074394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:14.074408 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:45:14.074549 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:14.074551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:14.074597 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:14.074698 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:14.075195 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:14.075797 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:14.076661 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:14.076904 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:14.076985 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:14.079688 (+ 2703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:43839: BlockingRecv error: recv got EOF from 127.31.237.129:43839 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":64,"threads_started":1}
Aug 08 10:45:14 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:14 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:14.280041 14577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:14.274175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:49291)
0808 10:45:14.274567 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:14.274580 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:14.274703 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:14.274975 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:14.274977 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:14.275168 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:14.275320 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:14.275327 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:14.276173 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:14.276176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:14.276853 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:14.276863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:14.277333 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:14.277335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:14.277401 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:14.277893 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:14.277904 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:14.279909 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":120,"threads_started":1}
W20260808 10:45:14.280283 14575 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:14.274255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:49291 (local address 127.31.237.190:33821)
0808 10:45:14.274414 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:14.274418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:14.274682 (+ 264us) server_negotiation.cc:407] Connection header received
0808 10:45:14.274804 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:14.274806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:14.274858 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:14.274936 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:14.275484 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:14.276046 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:14.276987 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:14.277223 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:14.277306 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:14.280167 (+ 2861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:49291: BlockingRecv error: recv got EOF from 127.31.237.130:49291 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 10:45:14 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:14 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:14.480768 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:14.474437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:36521)
0808 10:45:14.474790 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:14.474806 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:14.474935 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:14.475199 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:14.475201 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:14.475388 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:14.475482 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:14.475488 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:14.476315 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:14.476318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:14.477034 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:14.477045 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:14.477518 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:14.477521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:14.477600 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:14.478260 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:14.478273 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:14.480576 (+ 2303us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":153,"threads_started":1}
W20260808 10:45:14.480825 14575 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:14.474502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:36521 (local address 127.31.237.190:33821)
0808 10:45:14.474685 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:14.474690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:14.474911 (+ 221us) server_negotiation.cc:407] Connection header received
0808 10:45:14.475044 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:14.475046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:14.475098 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:14.475162 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:14.475584 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:14.476192 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:14.477149 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:14.477381 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:14.477463 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:14.480699 (+ 3236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:36521: BlockingRecv error: recv got EOF from 127.31.237.131:36521 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 10:45:15 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:15 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:15.086297 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:15.080611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:39909)
0808 10:45:15.080943 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:15.080959 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:15.081069 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:15.081366 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:15.081369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:15.081563 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:15.081665 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:15.081671 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:15.082512 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:15.082515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:15.083216 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:15.083227 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:15.083675 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:15.083677 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:15.083766 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:15.084300 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:15.084312 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:15.086170 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":86,"threads_started":1}
W20260808 10:45:15.086454 14580 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:15.080732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:39909 (local address 127.31.237.190:33821)
0808 10:45:15.080995 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:15.080998 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:15.081012 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:45:15.081165 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:15.081168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:15.081214 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:15.081288 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:15.081837 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:15.082393 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:15.083338 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:15.083574 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:15.083658 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:15.086314 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:39909: BlockingRecv error: recv got EOF from 127.31.237.129:39909 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":57,"threads_started":1}
Aug 08 10:45:15 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:15 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:15.286791 14582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:15.281137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:47723)
0808 10:45:15.281492 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:15.281507 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:15.281677 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:15.282037 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:15.282040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:15.282283 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:15.282414 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:15.282421 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:15.283183 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:15.283186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:15.283740 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:15.283748 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:15.284177 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:15.284179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:15.284241 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:15.284737 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:15.284749 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:15.286661 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":116,"threads_started":1}
W20260808 10:45:15.287045 14580 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:15.281254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:47723 (local address 127.31.237.190:33821)
0808 10:45:15.281430 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:15.281435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:15.281646 (+ 211us) server_negotiation.cc:407] Connection header received
0808 10:45:15.281772 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:15.281776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:15.281854 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:15.281971 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:15.282530 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:15.283085 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:15.283845 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:15.284077 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:15.284153 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:15.286773 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:47723: BlockingRecv error: recv got EOF from 127.31.237.130:47723 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 10:45:15 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:15 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:15.487123 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:15.481694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:59545)
0808 10:45:15.482007 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:15.482022 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:15.482125 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:15.482408 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:15.482411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:15.482597 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:15.482775 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:15.482781 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:15.483513 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:15.483516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:15.484067 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:15.484075 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:15.484475 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:15.484478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:15.484538 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:15.485068 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:15.485080 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:15.486993 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":123,"threads_started":1}
W20260808 10:45:15.487280 14580 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:15.481781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:59545 (local address 127.31.237.190:33821)
0808 10:45:15.481929 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:15.481933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:15.482093 (+ 160us) server_negotiation.cc:407] Connection header received
0808 10:45:15.482208 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:15.482211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:15.482258 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:15.482327 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:15.482899 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:15.483420 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:15.484168 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:15.484380 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:15.484454 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:15.487154 (+ 2700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:59545: BlockingRecv error: recv got EOF from 127.31.237.131:59545 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 10:45:16 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:16 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:16.093641 14586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:16.087337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:50537)
0808 10:45:16.087628 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:16.087642 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:16.087727 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:16.088010 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:16.088012 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:16.088204 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:16.088306 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:16.088311 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:16.089303 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:16.089305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:16.089906 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:16.089916 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:16.090541 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:16.090544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:16.090686 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:16.091369 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:16.091382 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:16.093519 (+ 2137us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":108,"threads_started":1}
W20260808 10:45:16.093833 14587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:16.087452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:50537 (local address 127.31.237.190:33821)
0808 10:45:16.087703 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:16.087707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:16.087721 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:45:16.087811 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:16.087813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:16.087859 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:16.087966 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:16.088443 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:16.089192 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:16.090039 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:16.090398 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:16.090536 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:16.093680 (+ 3144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:50537: BlockingRecv error: recv got EOF from 127.31.237.129:50537 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":96,"threads_started":1}
Aug 08 10:45:16 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:16 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:16.294052 14588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:16.287724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:54613)
0808 10:45:16.288159 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:16.288178 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:16.288353 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:16.288635 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:16.288638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:16.288882 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:16.288983 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:16.288990 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:16.289864 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:16.289869 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:16.290472 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:16.290482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:16.291015 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:16.291018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:16.291097 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:16.291922 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:16.291935 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:16.293931 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":178,"threads_started":1}
W20260808 10:45:16.294255 14587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:16.287977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:54613 (local address 127.31.237.190:33821)
0808 10:45:16.288161 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:16.288166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:16.288284 (+ 118us) server_negotiation.cc:407] Connection header received
0808 10:45:16.288462 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:16.288465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:16.288518 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:16.288608 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:16.289114 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:16.289743 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:16.290631 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:16.290882 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:16.291007 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:16.294105 (+ 3098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:54613: BlockingRecv error: recv got EOF from 127.31.237.130:54613 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 08 10:45:16 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:16 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:45:16.495743 14587 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:16.488251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:40015 (local address 127.31.237.190:33821)
0808 10:45:16.488383 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:16.488388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:16.488696 (+ 308us) server_negotiation.cc:407] Connection header received
0808 10:45:16.488853 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:16.488857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:16.488952 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:16.489036 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:16.489902 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:16.490436 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:16.491908 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:16.492207 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:16.492289 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:16.495646 (+ 3357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:40015: BlockingRecv error: recv got EOF from 127.31.237.131:40015 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:45:16.495761 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:16.488166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:40015)
0808 10:45:16.488589 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:16.488612 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:16.488748 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:16.489095 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:16.489273 (+ 178us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:16.489547 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:16.489731 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:16.489742 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:16.490551 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:16.490557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:16.491766 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:16.491780 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:16.492321 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:16.492327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:16.492457 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:16.493088 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:16.493109 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:16.495516 (+ 2407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":131,"threads_started":1}
Aug 08 10:45:17 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:17 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:17.103713 14590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:17.094735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:47607)
0808 10:45:17.095008 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:17.095022 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:17.095128 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:17.095348 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:17.095350 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:17.095533 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:17.095632 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:17.095636 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:17.096391 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:17.096394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:17.096943 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:17.096950 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:17.097388 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:17.097390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:17.097452 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:17.101698 (+ 4246us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:17.101711 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:17.103594 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":96,"threads_started":1}
W20260808 10:45:17.103942 14591 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:17.094785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:47607 (local address 127.31.237.190:33821)
0808 10:45:17.095034 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:17.095038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:17.095076 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:45:17.095213 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:17.095216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:17.095260 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:17.095330 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:17.095734 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:17.096284 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:17.097057 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:17.097284 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:17.097364 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:17.103743 (+ 6379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:47607: BlockingRecv error: recv got EOF from 127.31.237.129:47607 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":90,"threads_started":1}
Aug 08 10:45:17 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:17 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:17.301046 14593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:17.294952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:38285)
0808 10:45:17.295256 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:17.295270 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:17.295384 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:17.295633 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:17.295635 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:17.295828 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:17.295931 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:17.295937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:17.296679 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:17.296682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:17.297244 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:17.297253 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:17.297849 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:17.297853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:17.297943 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:17.298599 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:17.298649 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:17.300887 (+ 2238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":122,"threads_started":1}
W20260808 10:45:17.301231 14591 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:17.295050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:38285 (local address 127.31.237.190:33821)
0808 10:45:17.295192 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:17.295196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:17.295339 (+ 143us) server_negotiation.cc:407] Connection header received
0808 10:45:17.295468 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:17.295470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:17.295516 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:17.295595 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:17.296032 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:17.296573 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:17.297395 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:17.297725 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:17.297848 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:17.301043 (+ 3195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:38285: BlockingRecv error: recv got EOF from 127.31.237.130:38285 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 10:45:17 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:17 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:17.502336 14594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:17.496591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:37585)
0808 10:45:17.496884 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:17.496902 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:17.497026 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:17.497310 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:17.497313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:17.497538 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:17.497633 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:17.497639 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:17.498425 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:17.498428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:17.499083 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:17.499092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:17.499602 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:17.499606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:17.499692 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:17.500308 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:17.500331 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:17.502203 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":88,"threads_started":1}
W20260808 10:45:17.502476 14591 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:17.496697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:37585 (local address 127.31.237.190:33821)
0808 10:45:17.496841 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:17.496845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:17.496977 (+ 132us) server_negotiation.cc:407] Connection header received
0808 10:45:17.497132 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:17.497136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:17.497194 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:17.497281 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:17.497750 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:17.498297 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:17.499212 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:17.499476 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:17.499597 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:17.502350 (+ 2753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:37585: BlockingRecv error: recv got EOF from 127.31.237.131:37585 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 10:45:18 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:18 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:18.110553 14596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:18.104599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:32967)
0808 10:45:18.105104 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:18.105119 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:18.105276 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:18.105664 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:18.105668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:18.105886 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:18.105982 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:18.105988 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:18.106877 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:18.106883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:18.107462 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:18.107471 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:18.107966 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:18.107970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:18.108067 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:18.108567 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:18.108580 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:18.110434 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":129,"threads_started":1}
W20260808 10:45:18.110796 14598 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:18.104654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:32967 (local address 127.31.237.190:33821)
0808 10:45:18.104955 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:18.104962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:18.105204 (+ 242us) server_negotiation.cc:407] Connection header received
0808 10:45:18.105450 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:18.105456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:18.105518 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:18.105619 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:18.106110 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:18.106720 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:18.107587 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:18.107837 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:18.107945 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:18.110573 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:32967: BlockingRecv error: recv got EOF from 127.31.237.129:32967 (error 108)
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":120,"threads_started":1}
Aug 08 10:45:18 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:18 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:18.307238 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:18.301915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:48719)
0808 10:45:18.302248 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:18.302266 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:18.302387 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:18.302651 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:18.302654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:18.302860 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:18.302962 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:18.302968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:18.303685 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:18.303688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:18.304227 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:18.304235 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:18.304652 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:18.304655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:18.304716 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:18.305214 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:18.305226 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:18.307121 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":110,"threads_started":1}
W20260808 10:45:18.307425 14598 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:18.301967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:48719 (local address 127.31.237.190:33821)
0808 10:45:18.302140 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:18.302146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:18.302349 (+ 203us) server_negotiation.cc:407] Connection header received
0808 10:45:18.302456 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:18.302458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:18.302505 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:18.302572 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:18.303067 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:18.303591 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:18.304328 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:18.304548 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:18.304628 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:18.307262 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:48719: BlockingRecv error: recv got EOF from 127.31.237.130:48719 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 10:45:18 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:18 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:18.508659 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:18.503117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:56459)
0808 10:45:18.503446 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:18.503462 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:18.503570 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:18.503847 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:18.503850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:18.504041 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:18.504137 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:18.504143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:18.504862 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:18.504865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:18.505409 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:18.505416 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:18.505853 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:18.505856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:18.505917 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:18.506413 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:18.506432 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:18.508536 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":117,"threads_started":1}
W20260808 10:45:18.508921 14598 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:18.503234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:56459 (local address 127.31.237.190:33821)
0808 10:45:18.503371 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:18.503375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:18.503540 (+ 165us) server_negotiation.cc:407] Connection header received
0808 10:45:18.503651 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:18.503653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:18.503703 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:18.503785 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:18.504237 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:18.504755 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:18.505516 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:18.505735 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:18.505813 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:18.508791 (+ 2978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:56459: BlockingRecv error: recv got EOF from 127.31.237.131:56459 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 10:45:19 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:19 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:19.116899 14601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:19.111405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:32945)
0808 10:45:19.111681 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:19.111698 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:19.111803 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:19.112057 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:19.112060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:19.112246 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:19.112345 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:19.112350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:19.113167 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:19.113170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:19.113735 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:19.113744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:19.114229 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:19.114232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:19.114302 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:19.114941 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:19.114953 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:19.116777 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":71,"threads_started":1}
W20260808 10:45:19.117067 14607 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:19.111562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:32945 (local address 127.31.237.190:33821)
0808 10:45:19.111832 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:19.111835 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:19.111850 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:45:19.111887 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:19.111890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:19.111933 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:19.112019 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:19.112469 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:19.113038 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:19.113868 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:19.114096 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:19.114180 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:19.116927 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:32945: BlockingRecv error: recv got EOF from 127.31.237.129:32945 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":77,"threads_started":1}
Aug 08 10:45:19 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:19 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:19.313431 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:19.308015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:50031)
0808 10:45:19.308311 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:19.308326 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:19.308429 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:19.308638 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:19.308639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:19.308819 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:19.308935 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:19.308941 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:19.309786 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:19.309788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:19.310341 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:19.310348 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:19.310806 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:19.310809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:19.310872 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:19.311475 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:19.311488 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:19.313313 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":119,"threads_started":1}
W20260808 10:45:19.313627 14607 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:19.308091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:50031 (local address 127.31.237.190:33821)
0808 10:45:19.308270 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:19.308275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:19.308397 (+ 122us) server_negotiation.cc:407] Connection header received
0808 10:45:19.308510 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:19.308512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:19.308560 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:19.308626 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:19.309033 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:19.309695 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:19.310442 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:19.310720 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:19.310798 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:19.313459 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:50031: BlockingRecv error: recv got EOF from 127.31.237.130:50031 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 08 10:45:19 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:19 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:19.515010 14609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:19.509501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:48397)
0808 10:45:19.509814 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:19.509835 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:19.509940 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:19.510193 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:19.510195 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:19.510389 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:19.510488 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:19.510494 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:19.511214 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:19.511217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:19.511773 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:19.511782 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:19.512171 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:19.512174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:19.512232 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:19.512704 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:19.512726 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:19.514821 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":90,"threads_started":1}
W20260808 10:45:19.515056 14607 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:19.509595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:48397 (local address 127.31.237.190:33821)
0808 10:45:19.509747 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:19.509751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:19.509908 (+ 157us) server_negotiation.cc:407] Connection header received
0808 10:45:19.510021 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:19.510024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:19.510072 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:19.510145 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:19.510591 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:19.511120 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:19.511872 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:19.512076 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:19.512148 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:19.514953 (+ 2805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:48397: BlockingRecv error: recv got EOF from 127.31.237.131:48397 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 10:45:20 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:20 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:20.123406 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:20.117771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:32813)
0808 10:45:20.118101 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:20.118116 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:20.118209 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:20.118456 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:20.118459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:20.118722 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:20.118827 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:20.118833 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:20.119570 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:20.119573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:20.120147 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:20.120155 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:20.120581 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:20.120584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:20.120647 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:20.121134 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:20.121145 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:20.123275 (+ 2130us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":94,"threads_started":1}
W20260808 10:45:20.123584 14614 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:20.117861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:32813 (local address 127.31.237.190:33821)
0808 10:45:20.118161 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:20.118165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:20.118177 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:45:20.118280 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:20.118282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:20.118323 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:20.118407 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:20.118939 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:20.119471 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:20.120251 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:20.120483 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:20.120559 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:20.123410 (+ 2851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:32813: BlockingRecv error: recv got EOF from 127.31.237.129:32813 (error 108)
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":130,"threads_started":1}
Aug 08 10:45:20 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:20 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:20.319559 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:20.314212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:33119)
0808 10:45:20.314520 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:20.314535 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:20.314669 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:20.314910 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:20.314913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:20.315099 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:20.315195 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:20.315200 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:20.315915 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:20.315918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:20.316478 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:20.316487 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:20.316882 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:20.316884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:20.316946 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:20.317440 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:20.317452 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:20.319382 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":143,"threads_started":1}
W20260808 10:45:20.319599 14614 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:20.314316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:33119 (local address 127.31.237.190:33821)
0808 10:45:20.314469 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:20.314473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:20.314650 (+ 177us) server_negotiation.cc:407] Connection header received
0808 10:45:20.314747 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:20.314749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:20.314795 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:20.314869 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:20.315292 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:20.315812 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:20.316575 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:20.316783 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:20.316857 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:20.319500 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:33119: BlockingRecv error: recv got EOF from 127.31.237.130:33119 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 10:45:20 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:20 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:20.521711 14617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:20.515899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:37837)
0808 10:45:20.516216 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:20.516230 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:20.516322 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:20.516592 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:20.516595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:20.516782 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:20.516900 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:20.516906 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:20.517799 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:20.517804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:20.518709 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:20.518721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:20.519157 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:20.519160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:20.519221 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:20.519712 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:20.519723 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:20.521591 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":133,"threads_started":1}
W20260808 10:45:20.521849 14614 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:20.516148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:37837 (local address 127.31.237.190:33821)
0808 10:45:20.516296 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:20.516299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:20.516314 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:45:20.516424 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:20.516427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:20.516474 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:20.516549 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:20.517009 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:20.517665 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:20.518831 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:20.519055 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:20.519139 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:20.521727 (+ 2588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:37837: BlockingRecv error: recv got EOF from 127.31.237.131:37837 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 10:45:21 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:21 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:21.131176 14619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:21.124496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:46785)
0808 10:45:21.124841 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:21.124861 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:21.124967 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:21.125290 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:21.125292 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:21.125563 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:21.125681 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:21.125687 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:21.126587 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:21.126590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:21.127203 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:21.127213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:21.127757 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:21.127760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:21.127847 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:21.128448 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:21.128460 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:21.130997 (+ 2537us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":93,"threads_started":1}
W20260808 10:45:21.131464 14620 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:21.124590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:46785 (local address 127.31.237.190:33821)
0808 10:45:21.124889 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:21.124892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:21.124932 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:45:21.125094 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:21.125097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:21.125145 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:21.125221 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:21.125849 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:21.126470 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:21.127348 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:21.127620 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:21.127717 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:21.131157 (+ 3440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:46785: BlockingRecv error: recv got EOF from 127.31.237.129:46785 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":105,"threads_started":1}
Aug 08 10:45:21 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:21 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:21.326462 14621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:21.320470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:40915)
0808 10:45:21.320876 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:21.320891 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:21.321034 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:21.321330 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:21.321333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:21.321557 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:21.321653 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:21.321659 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:21.322456 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:21.322461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:21.323084 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:21.323093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:21.323569 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:21.323572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:21.323659 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:21.324217 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:21.324231 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:21.326338 (+ 2107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":90,"threads_started":1}
W20260808 10:45:21.326676 14620 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:21.320526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:40915 (local address 127.31.237.190:33821)
0808 10:45:21.320722 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:21.320727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:21.320975 (+ 248us) server_negotiation.cc:407] Connection header received
0808 10:45:21.321158 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:21.321160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:21.321207 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:21.321300 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:21.321783 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:21.322335 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:21.323223 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:21.323450 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:21.323561 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:21.326515 (+ 2954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:40915: BlockingRecv error: recv got EOF from 127.31.237.130:40915 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 10:45:21 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:21 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:21.528632 14622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:21.522702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:37361)
0808 10:45:21.523028 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:21.523045 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:21.523184 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:21.523482 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:21.523485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:21.523708 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:21.523803 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:21.523809 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:21.524586 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:21.524589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:21.525168 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:21.525176 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:21.525671 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:21.525674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:21.525762 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:21.526301 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:21.526313 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:21.528424 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":104,"threads_started":1}
W20260808 10:45:21.528676 14620 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:21.522888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:37361 (local address 127.31.237.190:33821)
0808 10:45:21.523028 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:21.523035 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:21.523127 (+ 92us) server_negotiation.cc:407] Connection header received
0808 10:45:21.523293 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:21.523298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:21.523351 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:21.523455 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:21.523926 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:21.524467 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:21.525295 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:21.525549 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:21.525664 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:21.528558 (+ 2894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:37361: BlockingRecv error: recv got EOF from 127.31.237.131:37361 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 10:45:22 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:22 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:22.142887 14624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:22.132050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:36601)
0808 10:45:22.132323 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:22.132339 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:22.132429 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:22.132661 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:22.132663 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:22.132941 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:22.133066 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:22.133078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:22.134133 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:22.134137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:22.135154 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:22.135169 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:22.136090 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:22.136094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:22.136206 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:22.140428 (+ 4222us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:22.140445 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:22.142651 (+ 2206us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":95,"threads_started":1}
W20260808 10:45:22.142921 14626 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:22.132179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:36601 (local address 127.31.237.190:33821)
0808 10:45:22.132419 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:22.132423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:22.132439 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:45:22.132496 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:22.132498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:22.132541 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:22.132623 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:22.133216 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:22.133986 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:22.135301 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:22.135794 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:22.135900 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:22.142791 (+ 6891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:36601: BlockingRecv error: recv got EOF from 127.31.237.129:36601 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":87,"threads_started":1}
Aug 08 10:45:22 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:22 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:22.333705 14628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:22.327475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:59027)
0808 10:45:22.327861 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:22.327876 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:22.328035 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:22.328342 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:22.328346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:22.328569 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:22.328668 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:22.328675 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:22.329467 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:22.329473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:22.330033 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:22.330042 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:22.330538 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:22.330542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:22.330670 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:22.331387 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:22.331430 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:22.333587 (+ 2157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":80,"threads_started":1}
W20260808 10:45:22.333982 14626 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:22.327564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:59027 (local address 127.31.237.190:33821)
0808 10:45:22.327734 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:22.327739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:22.327968 (+ 229us) server_negotiation.cc:407] Connection header received
0808 10:45:22.328158 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:22.328161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:22.328220 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:22.328311 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:22.328797 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:22.329349 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:22.330162 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:22.330408 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:22.330518 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:22.333858 (+ 3340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:59027: BlockingRecv error: recv got EOF from 127.31.237.130:59027 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 10:45:22 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:22 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:22.535156 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:22.529468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:58725)
0808 10:45:22.529737 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:22.529751 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:22.529837 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:22.530143 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:22.530147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:22.530334 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:22.530435 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:22.530441 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:22.531430 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:22.531434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:22.532041 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:22.532051 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:22.532482 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:22.532485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:22.532548 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:22.533166 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:22.533179 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:22.535039 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":73,"threads_started":1}
W20260808 10:45:22.535288 14626 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:22.529602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:58725 (local address 127.31.237.190:33821)
0808 10:45:22.529772 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:22.529775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:22.529805 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:45:22.529967 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:22.529969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:22.530016 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:22.530076 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:22.530523 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:22.531290 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:22.532153 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:22.532379 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:22.532459 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:22.535167 (+ 2708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:58725: BlockingRecv error: recv got EOF from 127.31.237.131:58725 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 10:45:23 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:23 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:23.149698 14630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:23.143742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:41037)
0808 10:45:23.144065 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:23.144079 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:23.144178 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:23.144435 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:23.144437 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:23.144619 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:23.144720 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:23.144726 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:23.145460 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:23.145463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:23.146266 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:23.146277 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:23.146907 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:23.146911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:23.146996 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:23.147620 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:23.147641 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:23.149527 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":123,"threads_started":1}
W20260808 10:45:23.149816 14631 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:23.143812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:41037 (local address 127.31.237.190:33821)
0808 10:45:23.144118 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:23.144121 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:23.144136 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:45:23.144268 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:23.144270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:23.144316 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:23.144410 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:23.144821 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:23.145343 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:23.146403 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:23.146782 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:23.146900 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:23.149673 (+ 2773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:41037: BlockingRecv error: recv got EOF from 127.31.237.129:41037 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":107,"threads_started":1}
Aug 08 10:45:23 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:23 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:23.340975 14633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:23.334809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:55227)
0808 10:45:23.335128 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:23.335142 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:23.335309 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:23.335662 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:23.335667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:23.335952 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:23.336089 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:23.336096 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:23.336902 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:23.336905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:23.337516 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:23.337531 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:23.338111 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:23.338114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:23.338189 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:23.338763 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:23.338783 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:23.340809 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":87,"threads_started":1}
W20260808 10:45:23.341115 14631 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:23.334882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:55227 (local address 127.31.237.190:33821)
0808 10:45:23.335067 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:23.335074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:23.335233 (+ 159us) server_negotiation.cc:407] Connection header received
0808 10:45:23.335420 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:23.335424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:23.335480 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:23.335580 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:23.336211 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:23.336739 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:23.337661 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:23.337959 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:23.338054 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:23.340993 (+ 2939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:55227: BlockingRecv error: recv got EOF from 127.31.237.130:55227 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 10:45:23 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:23 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:23.542078 14637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:23.536172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:53057)
0808 10:45:23.536481 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:23.536496 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:23.536604 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:23.536854 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:23.536857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:23.537069 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:23.537165 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:23.537170 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:23.537997 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:23.538000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:23.538737 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:23.538749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:23.539354 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:23.539361 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:23.539482 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:23.539999 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:23.540010 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:23.541956 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":126,"threads_started":1}
W20260808 10:45:23.542229 14631 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:23.536249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:53057 (local address 127.31.237.190:33821)
0808 10:45:23.536419 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:23.536425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:23.536572 (+ 147us) server_negotiation.cc:407] Connection header received
0808 10:45:23.536690 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:23.536692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:23.536744 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:23.536815 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:23.537266 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:23.537875 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:23.538870 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:23.539199 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:23.539317 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:23.542101 (+ 2784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:53057: BlockingRecv error: recv got EOF from 127.31.237.131:53057 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 10:45:24 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:24 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:24.155989 14640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:24.150669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:42475)
0808 10:45:24.150967 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:24.150980 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:24.151070 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:24.151319 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:24.151321 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:24.151502 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:24.151597 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:24.151602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:24.152347 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:24.152349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:24.152920 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:24.152929 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:24.153331 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:24.153334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:24.153394 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:24.153862 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:24.153873 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:24.155824 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":90,"threads_started":1}
W20260808 10:45:24.156136 14642 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:24.150727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:42475 (local address 127.31.237.190:33821)
0808 10:45:24.151025 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:24.151028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:24.151042 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:45:24.151157 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:24.151160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:24.151205 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:24.151290 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:24.151694 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:24.152253 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:24.153020 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:24.153236 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:24.153331 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:24.156004 (+ 2673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:42475: BlockingRecv error: recv got EOF from 127.31.237.129:42475 (error 108)
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":131,"threads_started":1}
Aug 08 10:45:24 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:24 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:24.347967 14643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:24.341945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:60615)
0808 10:45:24.342350 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:24.342364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:24.342475 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:24.342753 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:24.342756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:24.342953 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:24.343051 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:24.343057 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:24.343844 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:24.343846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:24.344392 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:24.344399 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:24.344883 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:24.344886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:24.344957 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:24.345495 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:24.345541 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:24.347790 (+ 2249us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":217,"threads_started":1}
W20260808 10:45:24.348031 14642 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:24.342134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:60615 (local address 127.31.237.190:33821)
0808 10:45:24.342464 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:24.342469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:24.342486 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:45:24.342546 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:24.342549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:24.342594 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:24.342715 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:24.343162 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:24.343748 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:24.344502 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:24.344739 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:24.344842 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:24.347923 (+ 3081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:60615: BlockingRecv error: recv got EOF from 127.31.237.130:60615 (error 108)
Metrics: {"server-negotiator.queue_time_us":223}
Aug 08 10:45:24 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:24 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:24.548667 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:24.542989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:37851)
0808 10:45:24.543275 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:24.543290 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:24.543406 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:24.543629 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:24.543632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:24.543816 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:24.543917 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:24.543923 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:24.544645 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:24.544648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:24.545304 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:24.545313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:24.545778 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:24.545782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:24.545883 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:24.546507 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:24.546531 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:24.548494 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":116,"threads_started":1}
W20260808 10:45:24.548705 14642 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:24.543109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:37851 (local address 127.31.237.190:33821)
0808 10:45:24.543254 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:24.543258 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:24.543339 (+ 81us) server_negotiation.cc:407] Connection header received
0808 10:45:24.543474 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:24.543477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:24.543522 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:24.543593 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:24.544020 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:24.544550 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:24.545421 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:24.545657 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:24.545771 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:24.548606 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:37851: BlockingRecv error: recv got EOF from 127.31.237.131:37851 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 10:45:25 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:25 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:25.162974 14646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:25.156832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:42735)
0808 10:45:25.157146 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:25.157166 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:25.157284 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:25.157609 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:25.157612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:25.157873 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:25.157984 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:25.157991 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:25.158809 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:25.158812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:25.159434 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:25.159444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:25.159896 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:25.159898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:25.159963 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:25.160560 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:25.160579 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:25.162840 (+ 2261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":103,"threads_started":1}
W20260808 10:45:25.163203 14647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:25.156997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:42735 (local address 127.31.237.190:33821)
0808 10:45:25.157356 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:25.157360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:25.157373 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:45:25.157410 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:25.157412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:25.157454 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:25.157537 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:25.158131 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:25.158699 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:25.159547 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:25.159769 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:25.159847 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:25.163046 (+ 3199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:42735: BlockingRecv error: recv got EOF from 127.31.237.129:42735 (error 108)
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":85,"threads_started":1}
Aug 08 10:45:25 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:25 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:25.354342 14648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:25.348766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:48751)
0808 10:45:25.349073 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:25.349089 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:25.349206 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:25.349516 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:25.349518 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:25.349728 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:25.349839 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:25.349848 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:25.350638 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:25.350641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:25.351260 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:25.351270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:25.351721 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:25.351724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:25.351796 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:25.352337 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:25.352351 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:25.354200 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":73,"threads_started":1}
W20260808 10:45:25.354489 14647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:25.348946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:48751 (local address 127.31.237.190:33821)
0808 10:45:25.349103 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:25.349107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:25.349147 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:45:25.349295 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:25.349297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:25.349342 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:25.349468 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:25.349957 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:25.350508 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:25.351389 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:25.351611 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:25.351691 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:25.354348 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:48751: BlockingRecv error: recv got EOF from 127.31.237.130:48751 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 10:45:25 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:25 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.131@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:25.555371 14649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:25.549577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.131:54609)
0808 10:45:25.549872 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:25.549887 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:25.549991 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:25.550230 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:25.550232 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:25.550443 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:25.550539 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:25.550545 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:25.551337 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:25.551355 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:25.551901 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:25.551909 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:25.552320 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:25.552322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:25.552403 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:25.553006 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:25.553018 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:25.555198 (+ 2180us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":91,"threads_started":1}
W20260808 10:45:25.555414 14647 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:25.549672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:54609 (local address 127.31.237.190:33821)
0808 10:45:25.549862 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:25.549866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:25.549958 (+ 92us) server_negotiation.cc:407] Connection header received
0808 10:45:25.550073 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:25.550075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:25.550123 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:25.550191 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:25.550692 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:25.551245 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:25.552007 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:25.552220 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:25.552298 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:25.555313 (+ 3015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.131:54609: BlockingRecv error: recv got EOF from 127.31.237.131:54609 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 08 10:45:26 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:26 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.129@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:26.169884 14650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:26.163947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.129:51817)
0808 10:45:26.164262 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:26.164282 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:26.164415 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:26.164722 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:26.164724 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:26.164928 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:26.165032 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:26.165038 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:26.165850 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:26.165853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:26.166483 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:26.166492 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:26.166976 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:26.166980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:26.167053 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:26.167621 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:26.167634 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:26.169742 (+ 2108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":75,"threads_started":1}
W20260808 10:45:26.170065 14651 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:26.164105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:51817 (local address 127.31.237.190:33821)
0808 10:45:26.164388 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:26.164392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:26.164406 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:45:26.164514 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:26.164516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:26.164563 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:26.164644 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:26.165161 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:26.165720 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:26.166616 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:26.166862 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:26.166946 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:26.169915 (+ 2969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.129:51817: BlockingRecv error: recv got EOF from 127.31.237.129:51817 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":90,"threads_started":1}
Aug 08 10:45:26 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:45:26 dist-test-slave-twwt krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.237.130@KRBTEST.COM for oryx/127.31.237.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:45:26.364461 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:26.355695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33821 (local address 127.31.237.130:47355)
0808 10:45:26.356191 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:26.356218 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:26.356443 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:26.356872 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:26.356881 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:26.357261 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:26.357407 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:26.357415 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:26.358565 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:26.358582 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:26.359474 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:26.359494 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:26.360212 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:26.360218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:26.360337 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:26.361245 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:26.361259 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:26.364247 (+ 2988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.237.190:33821: Server oryx/127.31.237.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":138,"threads_started":1}
W20260808 10:45:26.364775 14651 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:26.355840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:47355 (local address 127.31.237.190:33821)
0808 10:45:26.356048 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:26.356056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:26.356352 (+ 296us) server_negotiation.cc:407] Connection header received
0808 10:45:26.356633 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:26.356638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:26.356710 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:26.356845 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:26.357581 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:26.358395 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:26.359666 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:26.360022 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:26.360189 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:26.364494 (+ 4305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.237.130:47355: BlockingRecv error: recv got EOF from 127.31.237.130:47355 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 10:45:26.455093 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 13914
I20260808 10:45:26.481680 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 14153
I20260808 10:45:26.501704 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 14346
I20260808 10:45:26.522300 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 13812
2026-08-08T10:45:26Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18453 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T10:45:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:45:26Z Disabled control of system clock
I20260808 10:45:26.578071 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:39021
--webserver_interface=127.31.237.190
--webserver_port=0
--builtin_ntp_servers=127.31.237.148:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:39021
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 10:45:26.781906 14657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:26.782421 14657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:26.782483 14657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:26.788622 14657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:45:26.788723 14657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:26.788763 14657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:26.788897 14657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:45:26.788976 14657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:45:26.797171 14657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:39021
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:39021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:26.799243 14657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:26.801636 14657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:26.815809 14670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:26.815855 14668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:26.815912 14664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:26.816135 14657 server_base.cc:1034] running on GCE node
I20260808 10:45:26.817365 14657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:26.819545 14657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:26.820749 14657 hybrid_clock.cc:648] HybridClock initialized: now 1786185926820711 us; error 55 us; skew 500 ppm
I20260808 10:45:26.823820 14657 webserver.cc:460] Webserver started at http://127.31.237.190:43059/ using document root <none> and password file <none>
I20260808 10:45:26.824765 14657 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:26.824985 14657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:26.825408 14657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:26.828892 14657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "78815ad671a34f8ea5b80276c8584257"
format_stamp: "Formatted at 2026-08-08 10:45:26 on dist-test-slave-twwt"
server_key: "095843b3c5b3580f13c61cab03527b23"
server_key_iv: "44dd5fe3d9945917e00cb9d536878624"
server_key_version: "encryptionkey@0"
I20260808 10:45:26.830041 14657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "78815ad671a34f8ea5b80276c8584257"
format_stamp: "Formatted at 2026-08-08 10:45:26 on dist-test-slave-twwt"
server_key: "095843b3c5b3580f13c61cab03527b23"
server_key_iv: "44dd5fe3d9945917e00cb9d536878624"
server_key_version: "encryptionkey@0"
I20260808 10:45:26.836977 14657 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260808 10:45:26.841351 14679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:26.843313 14657 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 10:45:26.843538 14657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "78815ad671a34f8ea5b80276c8584257"
format_stamp: "Formatted at 2026-08-08 10:45:26 on dist-test-slave-twwt"
server_key: "095843b3c5b3580f13c61cab03527b23"
server_key_iv: "44dd5fe3d9945917e00cb9d536878624"
server_key_version: "encryptionkey@0"
I20260808 10:45:26.843730 14657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:26.863166 14657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:26.864236 14657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:26.864588 14657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:26.875778 14657 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:39021
I20260808 10:45:26.875818 14765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:39021 every 8 connection(s)
I20260808 10:45:26.877282 14657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:45:26.880368 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 14657
I20260808 10:45:26.880558 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:45:26.880975 32694 external_mini_cluster.cc:1428] Setting key 23726999ef99722539ec368129785109
I20260808 10:45:26.882988 14766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:26.895059 14766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257: Bootstrap starting.
I20260808 10:45:26.899106 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:26.883230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44724 (local address 127.31.237.190:39021)
0808 10:45:26.884335 (+ 1105us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:26.884385 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:26.884455 (+ 70us) server_negotiation.cc:407] Connection header received
0808 10:45:26.886000 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:26.886090 (+ 90us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:26.887093 (+ 1003us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:26.887762 (+ 669us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:26.888610 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:26.891231 (+ 2621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:26.892616 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:26.893959 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:26.894225 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:26.895610 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:26.895628 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:26.895660 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:45:26.896449 (+ 789us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:26.896550 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:45:26.896603 (+ 53us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:26.896770 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:26.896936 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 10:45:26.897394 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":138,"threads_started":1}
I20260808 10:45:26.899544 14766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:26.900578 14766 log.cc:824] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:26.903964 14766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257: No bootstrap required, opened a new log
I20260808 10:45:26.909567 14766 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78815ad671a34f8ea5b80276c8584257" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39021 } }
I20260808 10:45:26.909978 14766 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:26.910109 14766 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78815ad671a34f8ea5b80276c8584257, State: Initialized, Role: FOLLOWER
I20260808 10:45:26.910825 14766 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78815ad671a34f8ea5b80276c8584257" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39021 } }
I20260808 10:45:26.911047 14766 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:45:26.911173 14766 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:45:26.911356 14766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:26.912782 14766 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78815ad671a34f8ea5b80276c8584257" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39021 } }
I20260808 10:45:26.913326 14766 leader_election.cc:302] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78815ad671a34f8ea5b80276c8584257; no voters:
I20260808 10:45:26.913837 14766 leader_election.cc:288] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:45:26.914031 14773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:45:26.914398 14773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [term 1 LEADER]: Becoming Leader. State: Replica: 78815ad671a34f8ea5b80276c8584257, State: Running, Role: LEADER
I20260808 10:45:26.915037 14773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78815ad671a34f8ea5b80276c8584257" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39021 } }
I20260808 10:45:26.915889 14766 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:45:26.917656 14775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78815ad671a34f8ea5b80276c8584257. Latest consensus state: current_term: 1 leader_uuid: "78815ad671a34f8ea5b80276c8584257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78815ad671a34f8ea5b80276c8584257" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39021 } } }
I20260808 10:45:26.917862 14775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:26.918520 14774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78815ad671a34f8ea5b80276c8584257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78815ad671a34f8ea5b80276c8584257" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39021 } } }
I20260808 10:45:26.918637 14792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:45:26.918771 14774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257 [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:26.924333 14792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:45:26.932354 14792 catalog_manager.cc:1347] Generated new cluster ID: cef3fe8ebf3444c5b3497a5d05a8891c
I20260808 10:45:26.932442 14792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:45:26.943267 14792 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:45:26.945050 14792 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:45:26.957381 14792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78815ad671a34f8ea5b80276c8584257: Generated new TSK 0
I20260808 10:45:26.958743 14792 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:45:26.971094 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39021
--builtin_ntp_servers=127.31.237.148:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 10:45:27.159458 14808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:27.159822 14808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:27.159868 14808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:27.165695 14808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:27.165782 14808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:27.165903 14808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:45:27.173051 14808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:27.174937 14808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:27.177352 14808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:27.193481 14820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:27.193495 14816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:27.193758 14814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:27.193971 14808 server_base.cc:1034] running on GCE node
I20260808 10:45:27.194895 14808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:27.195809 14808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:27.197021 14808 hybrid_clock.cc:648] HybridClock initialized: now 1786185927196996 us; error 47 us; skew 500 ppm
I20260808 10:45:27.203788 14808 webserver.cc:460] Webserver started at http://127.31.237.129:44627/ using document root <none> and password file <none>
I20260808 10:45:27.204721 14808 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:27.204800 14808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:27.205103 14808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:27.208474 14808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "2992cf8e67d8460b900836b1cf9c1f56"
format_stamp: "Formatted at 2026-08-08 10:45:27 on dist-test-slave-twwt"
server_key: "bbe33227690f1a13297e5c8b5ba9eacb"
server_key_iv: "299a1d6ef5a6eb9a8d406f9f0a2def9f"
server_key_version: "encryptionkey@0"
I20260808 10:45:27.209336 14808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "2992cf8e67d8460b900836b1cf9c1f56"
format_stamp: "Formatted at 2026-08-08 10:45:27 on dist-test-slave-twwt"
server_key: "bbe33227690f1a13297e5c8b5ba9eacb"
server_key_iv: "299a1d6ef5a6eb9a8d406f9f0a2def9f"
server_key_version: "encryptionkey@0"
I20260808 10:45:27.215476 14808 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260808 10:45:27.219745 14832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:27.221029 14808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260808 10:45:27.221132 14808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "2992cf8e67d8460b900836b1cf9c1f56"
format_stamp: "Formatted at 2026-08-08 10:45:27 on dist-test-slave-twwt"
server_key: "bbe33227690f1a13297e5c8b5ba9eacb"
server_key_iv: "299a1d6ef5a6eb9a8d406f9f0a2def9f"
server_key_version: "encryptionkey@0"
I20260808 10:45:27.221227 14808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:27.240139 14808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:27.240828 14808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:27.240985 14808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:27.241552 14808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:27.242476 14808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:27.242527 14808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:27.242568 14808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:27.242584 14808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:27.252398 14808 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:45957
I20260808 10:45:27.252419 15017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:45957 every 8 connection(s)
I20260808 10:45:27.253366 14808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:45:27.263520 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 14808
I20260808 10:45:27.263548 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:27.255224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:33793 (local address 127.31.237.190:39021)
0808 10:45:27.255382 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:27.255386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:27.256474 (+ 1088us) server_negotiation.cc:407] Connection header received
0808 10:45:27.258049 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:27.258053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:27.258114 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:27.258240 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:27.259281 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:27.259935 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:27.261359 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:27.261622 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:27.261743 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:27.262987 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:27.262992 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:27.262995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:45:27.263015 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:27.263034 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:45:27.263043 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:27.263124 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:27.263366 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 10:45:27.263431 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:45:27.263664 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:45:27.264000 32694 external_mini_cluster.cc:1428] Setting key 91c9180d43253039035476a17183c0e1
I20260808 10:45:27.264796 15023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:27.255641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39021 (local address 127.31.237.129:33793)
0808 10:45:27.256317 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:27.256362 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:27.257734 (+ 1372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:27.258432 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:27.258440 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:27.258537 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:27.259093 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:27.259109 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:27.260080 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:27.260084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:27.261217 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:27.261230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:27.261798 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:27.261804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:27.261980 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:27.262690 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:45:27.262692 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:27.262743 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:45:27.262812 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:27.263141 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:27.263154 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:27.263165 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 10:45:27.263347 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 10:45:27.263691 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":127,"threads_started":1}
I20260808 10:45:27.266302 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39021
--builtin_ntp_servers=127.31.237.148:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 10:45:27.266307 15019 heartbeater.cc:347] Connected to a master server at 127.31.237.190:39021
I20260808 10:45:27.266563 15019 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:27.267210 15019 heartbeater.cc:511] Master 127.31.237.190:39021 requested a full tablet report, sending...
I20260808 10:45:27.268975 14703 ts_manager.cc:194] Registered new tserver with Master: 2992cf8e67d8460b900836b1cf9c1f56 (127.31.237.129:45957)
I20260808 10:45:27.270673 14703 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.237.129:33793
W20260808 10:45:27.382968 15025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:27.383236 15025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:27.383275 15025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:27.387274 15025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:27.387336 15025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:27.387391 15025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:45:27.392953 15025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:27.394286 15025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:27.395761 15025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:27.405505 15036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:27.405519 15033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:27.405544 15031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:27.405624 15025 server_base.cc:1034] running on GCE node
I20260808 10:45:27.406198 15025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:27.406948 15025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:27.408130 15025 hybrid_clock.cc:648] HybridClock initialized: now 1786185927408099 us; error 52 us; skew 500 ppm
I20260808 10:45:27.409996 15025 webserver.cc:460] Webserver started at http://127.31.237.130:45537/ using document root <none> and password file <none>
I20260808 10:45:27.410681 15025 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:27.410773 15025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:27.411002 15025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:27.413112 15025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "f8a6843b55744f039c36a963210d3041"
format_stamp: "Formatted at 2026-08-08 10:45:27 on dist-test-slave-twwt"
server_key: "6faf3b2633b9ed19ce97d726cfefea83"
server_key_iv: "5ddf64f8c47d12ce4d3ae219d8d829b3"
server_key_version: "encryptionkey@0"
I20260808 10:45:27.413734 15025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "f8a6843b55744f039c36a963210d3041"
format_stamp: "Formatted at 2026-08-08 10:45:27 on dist-test-slave-twwt"
server_key: "6faf3b2633b9ed19ce97d726cfefea83"
server_key_iv: "5ddf64f8c47d12ce4d3ae219d8d829b3"
server_key_version: "encryptionkey@0"
I20260808 10:45:27.417747 15025 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 10:45:27.420527 15044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:27.421725 15025 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:45:27.421862 15025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "f8a6843b55744f039c36a963210d3041"
format_stamp: "Formatted at 2026-08-08 10:45:27 on dist-test-slave-twwt"
server_key: "6faf3b2633b9ed19ce97d726cfefea83"
server_key_iv: "5ddf64f8c47d12ce4d3ae219d8d829b3"
server_key_version: "encryptionkey@0"
I20260808 10:45:27.421977 15025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:27.455251 15025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:27.456035 15025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:27.456251 15025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:27.456928 15025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:27.457954 15025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:27.458024 15025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:27.458091 15025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:27.458137 15025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:27.467080 15025 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:41897
I20260808 10:45:27.467140 15219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:41897 every 8 connection(s)
I20260808 10:45:27.468046 15025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:45:27.474689 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 15025
I20260808 10:45:27.474810 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:45:27.475082 32694 external_mini_cluster.cc:1428] Setting key 4585110c1993c733e4bdfd0ce5c5c0a9
I20260808 10:45:27.477192 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39021
--builtin_ntp_servers=127.31.237.148:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 10:45:27.477355 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:27.469867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:35051 (local address 127.31.237.190:39021)
0808 10:45:27.469994 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:27.469998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:27.471092 (+ 1094us) server_negotiation.cc:407] Connection header received
0808 10:45:27.472428 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:27.472432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:27.472479 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:27.472556 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:27.473402 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:27.473915 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:27.475205 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:27.475473 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:27.475592 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:27.476847 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:27.476849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:27.476852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:45:27.476863 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:27.476885 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:45:27.476893 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:27.476953 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:27.477148 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 10:45:27.477198 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:45:27.478490 15225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:27.470247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39021 (local address 127.31.237.130:35051)
0808 10:45:27.470935 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:27.470978 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:27.472238 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:27.472703 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:27.472709 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:27.472799 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:27.473285 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:27.473297 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:27.474019 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:27.474022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:27.475081 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:27.475091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:27.475674 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:27.475679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:27.475881 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:27.476557 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:45:27.476559 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:27.476611 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:45:27.476691 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:27.476983 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:27.476995 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:27.477005 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 10:45:27.477143 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 10:45:27.477456 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":115,"threads_started":1}
I20260808 10:45:27.479604 15221 heartbeater.cc:347] Connected to a master server at 127.31.237.190:39021
I20260808 10:45:27.479848 15221 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:27.480362 15221 heartbeater.cc:511] Master 127.31.237.190:39021 requested a full tablet report, sending...
I20260808 10:45:27.481386 14703 ts_manager.cc:194] Registered new tserver with Master: f8a6843b55744f039c36a963210d3041 (127.31.237.130:41897)
I20260808 10:45:27.482017 14703 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.237.130:35051
W20260808 10:45:27.594309 15226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:27.594705 15226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:27.594811 15226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:27.599170 15226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:27.599257 15226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:27.599354 15226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:45:27.604377 15226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:27.605522 15226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:27.607014 15226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:27.617170 15234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:27.617236 15237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:27.617331 15226 server_base.cc:1034] running on GCE node
W20260808 10:45:27.617164 15235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:27.617892 15226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:27.618726 15226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:27.619907 15226 hybrid_clock.cc:648] HybridClock initialized: now 1786185927619885 us; error 31 us; skew 500 ppm
I20260808 10:45:27.621940 15226 webserver.cc:460] Webserver started at http://127.31.237.131:34239/ using document root <none> and password file <none>
I20260808 10:45:27.622589 15226 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:27.622740 15226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:27.622958 15226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:27.625198 15226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "074d1b5bceae4687963b21df60603845"
format_stamp: "Formatted at 2026-08-08 10:45:27 on dist-test-slave-twwt"
server_key: "a4677014b627e77d159c0d725f9f3de6"
server_key_iv: "6413c71b7463846c3b2e5d465c6bcc88"
server_key_version: "encryptionkey@0"
I20260808 10:45:27.625741 15226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "074d1b5bceae4687963b21df60603845"
format_stamp: "Formatted at 2026-08-08 10:45:27 on dist-test-slave-twwt"
server_key: "a4677014b627e77d159c0d725f9f3de6"
server_key_iv: "6413c71b7463846c3b2e5d465c6bcc88"
server_key_version: "encryptionkey@0"
I20260808 10:45:27.629716 15226 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 10:45:27.632354 15246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:27.633663 15226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:45:27.633785 15226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "074d1b5bceae4687963b21df60603845"
format_stamp: "Formatted at 2026-08-08 10:45:27 on dist-test-slave-twwt"
server_key: "a4677014b627e77d159c0d725f9f3de6"
server_key_iv: "6413c71b7463846c3b2e5d465c6bcc88"
server_key_version: "encryptionkey@0"
I20260808 10:45:27.633900 15226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:27.648486 15226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:27.649212 15226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:27.649415 15226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:27.650049 15226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:27.651110 15226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:27.651162 15226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:27.651226 15226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:27.651258 15226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:27.660427 15226 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:38457
I20260808 10:45:27.660446 15428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:38457 every 8 connection(s)
I20260808 10:45:27.661335 15226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:45:27.665030 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 15226
I20260808 10:45:27.665176 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:45:27.665500 32694 external_mini_cluster.cc:1428] Setting key 8e4d5a3e9c0dcd573fb6275875b517cc
I20260808 10:45:27.670969 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:27.663362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:52451 (local address 127.31.237.190:39021)
0808 10:45:27.663504 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:27.663508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:27.664653 (+ 1145us) server_negotiation.cc:407] Connection header received
0808 10:45:27.666328 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:27.666332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:27.666398 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:27.666496 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:27.667446 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:27.667973 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:27.669108 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:27.669326 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:27.669444 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:27.670450 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:27.670454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:27.670457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:45:27.670475 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:27.670493 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:45:27.670500 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:27.670574 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:27.670767 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 10:45:27.670828 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:45:27.671795 15431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:27.663818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39021 (local address 127.31.237.131:52451)
0808 10:45:27.664508 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:27.664548 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:27.666123 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:27.666719 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:27.666725 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:27.666811 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:27.667307 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:27.667319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:27.668106 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:27.668108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:27.668985 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:27.668995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:27.669481 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:27.669485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:27.669647 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:27.670201 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:45:27.670203 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:27.670241 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:45:27.670304 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:27.670576 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:27.670588 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:27.670595 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:45:27.670767 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 10:45:27.671055 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":127,"threads_started":1}
I20260808 10:45:27.672881 15429 heartbeater.cc:347] Connected to a master server at 127.31.237.190:39021
I20260808 10:45:27.673141 15429 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:27.673687 15429 heartbeater.cc:511] Master 127.31.237.190:39021 requested a full tablet report, sending...
I20260808 10:45:27.674928 14703 ts_manager.cc:194] Registered new tserver with Master: 074d1b5bceae4687963b21df60603845 (127.31.237.131:38457)
I20260808 10:45:27.675531 14703 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.237.131:52451
I20260808 10:45:27.679697 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:45:27.682963 14771 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:27.681785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44730 (local address 127.31.237.190:39021)
0808 10:45:27.681941 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:27.681948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:27.682182 (+ 234us) server_negotiation.cc:407] Connection header received
0808 10:45:27.682348 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:27.682353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:27.682404 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:27.682503 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:27.682812 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44730: BlockingRecv error: recv got EOF from 127.0.0.1:44730 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:45:27.683112 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 14808
I20260808 10:45:27.712587 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 15025
I20260808 10:45:27.725286 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 15226
I20260808 10:45:27.746829 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 14657
2026-08-08T10:45:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1213 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T10:45:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:45:27Z Disabled control of system clock
I20260808 10:45:27.785465 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:44263
--webserver_interface=127.31.237.190
--webserver_port=0
--builtin_ntp_servers=127.31.237.148:44403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:44263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:45:27.904666 15443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:27.904942 15443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:27.904982 15443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:27.909005 15443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:45:27.909063 15443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:27.909080 15443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:27.909096 15443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:45:27.909137 15443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:45:27.913945 15443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:44263
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:44263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:27.915077 15443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:27.916487 15443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:27.925171 15453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:27.925180 15451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:27.925192 15456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:27.925217 15443 server_base.cc:1034] running on GCE node
I20260808 10:45:27.925900 15443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:27.927049 15443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:27.928242 15443 hybrid_clock.cc:648] HybridClock initialized: now 1786185927928225 us; error 27 us; skew 500 ppm
I20260808 10:45:27.930069 15443 webserver.cc:460] Webserver started at http://127.31.237.190:45241/ using document root <none> and password file <none>
I20260808 10:45:27.930714 15443 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:27.930805 15443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:27.931039 15443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:27.933153 15443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "4c92de1acf3d4375acf65d3243f12cea"
format_stamp: "Formatted at 2026-08-08 10:45:27 on dist-test-slave-twwt"
server_key: "94bd206d2a4bfd8f965999653e8ecd06"
server_key_iv: "743f90bf5d3e0693e0af3aa7be04607f"
server_key_version: "encryptionkey@0"
I20260808 10:45:27.933737 15443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "4c92de1acf3d4375acf65d3243f12cea"
format_stamp: "Formatted at 2026-08-08 10:45:27 on dist-test-slave-twwt"
server_key: "94bd206d2a4bfd8f965999653e8ecd06"
server_key_iv: "743f90bf5d3e0693e0af3aa7be04607f"
server_key_version: "encryptionkey@0"
I20260808 10:45:27.937526 15443 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:45:27.940080 15466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:27.941275 15443 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:45:27.941412 15443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "4c92de1acf3d4375acf65d3243f12cea"
format_stamp: "Formatted at 2026-08-08 10:45:27 on dist-test-slave-twwt"
server_key: "94bd206d2a4bfd8f965999653e8ecd06"
server_key_iv: "743f90bf5d3e0693e0af3aa7be04607f"
server_key_version: "encryptionkey@0"
I20260808 10:45:27.941537 15443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:27.946156 15443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:27.946775 15443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:27.946918 15443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:27.954860 15443 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:44263
I20260808 10:45:27.956203 15443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:45:27.958904 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 15443
I20260808 10:45:27.959039 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:45:27.959378 32694 external_mini_cluster.cc:1428] Setting key be970a470061d7a5bc73b34f14a4e72c
I20260808 10:45:27.960012 15551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:27.962450 15550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:44263 every 8 connection(s)
I20260808 10:45:27.967820 15551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea: Bootstrap starting.
I20260808 10:45:27.969280 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:27.963529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50692 (local address 127.31.237.190:44263)
0808 10:45:27.964824 (+ 1295us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:27.964846 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:27.964889 (+ 43us) server_negotiation.cc:407] Connection header received
0808 10:45:27.965794 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:27.965821 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:27.966287 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:27.966864 (+ 577us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:27.967939 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:27.967945 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:27.967962 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:45:27.967975 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:27.968048 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:45:27.968075 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:27.968229 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:27.968305 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 10:45:27.968596 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830,"thread_start_us":119,"threads_started":1}
I20260808 10:45:27.970264 15551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:27.971205 15551 log.cc:824] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:27.973173 15551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea: No bootstrap required, opened a new log
I20260808 10:45:27.976675 15551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c92de1acf3d4375acf65d3243f12cea" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44263 } }
I20260808 10:45:27.976907 15551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:27.976941 15551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c92de1acf3d4375acf65d3243f12cea, State: Initialized, Role: FOLLOWER
I20260808 10:45:27.977402 15551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c92de1acf3d4375acf65d3243f12cea" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44263 } }
I20260808 10:45:27.977542 15551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:45:27.977633 15551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:45:27.977734 15551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:27.978803 15551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c92de1acf3d4375acf65d3243f12cea" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44263 } }
I20260808 10:45:27.979115 15551 leader_election.cc:302] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c92de1acf3d4375acf65d3243f12cea; no voters:
I20260808 10:45:27.979389 15551 leader_election.cc:288] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:45:27.979537 15566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:45:27.979763 15566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [term 1 LEADER]: Becoming Leader. State: Replica: 4c92de1acf3d4375acf65d3243f12cea, State: Running, Role: LEADER
I20260808 10:45:27.980140 15566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c92de1acf3d4375acf65d3243f12cea" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44263 } }
I20260808 10:45:27.980660 15551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:45:27.981961 15567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c92de1acf3d4375acf65d3243f12cea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c92de1acf3d4375acf65d3243f12cea" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44263 } } }
I20260808 10:45:27.981961 15568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c92de1acf3d4375acf65d3243f12cea. Latest consensus state: current_term: 1 leader_uuid: "4c92de1acf3d4375acf65d3243f12cea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c92de1acf3d4375acf65d3243f12cea" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 44263 } } }
I20260808 10:45:27.982116 15567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:27.982116 15568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:27.982586 15578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:45:27.985500 15578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:45:27.991196 15578 catalog_manager.cc:1347] Generated new cluster ID: 865ba7d815364d8c9bcf04ac1d9e5f1e
I20260808 10:45:27.991266 15578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:45:28.008383 15578 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:45:28.008791 15578 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:45:28.015797 15578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c92de1acf3d4375acf65d3243f12cea: Generated new TSK 0
I20260808 10:45:28.016731 15578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:45:28.022138 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44263
--builtin_ntp_servers=127.31.237.148:44403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:45:28.140508 15589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:28.140800 15589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:28.140874 15589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:28.144936 15589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:28.145004 15589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:28.145084 15589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:45:28.149780 15589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:28.150924 15589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:28.152356 15589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:28.162976 15597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:28.162972 15595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:28.162976 15589 server_base.cc:1034] running on GCE node
W20260808 10:45:28.162997 15600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:28.163676 15589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:28.164358 15589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:28.165550 15589 hybrid_clock.cc:648] HybridClock initialized: now 1786185928165532 us; error 30 us; skew 500 ppm
I20260808 10:45:28.167510 15589 webserver.cc:460] Webserver started at http://127.31.237.129:39773/ using document root <none> and password file <none>
I20260808 10:45:28.168116 15589 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:28.168210 15589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:28.168450 15589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:28.170559 15589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "1ce587b874804707969828d739e85970"
format_stamp: "Formatted at 2026-08-08 10:45:28 on dist-test-slave-twwt"
server_key: "cdbf1cb3867d0a114f0ab2b510e82ad9"
server_key_iv: "ccaac9743b0d0c30bde0876e875b9ec8"
server_key_version: "encryptionkey@0"
I20260808 10:45:28.171191 15589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "1ce587b874804707969828d739e85970"
format_stamp: "Formatted at 2026-08-08 10:45:28 on dist-test-slave-twwt"
server_key: "cdbf1cb3867d0a114f0ab2b510e82ad9"
server_key_iv: "ccaac9743b0d0c30bde0876e875b9ec8"
server_key_version: "encryptionkey@0"
I20260808 10:45:28.174903 15589 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:45:28.177486 15614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:28.178753 15589 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:45:28.178875 15589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "1ce587b874804707969828d739e85970"
format_stamp: "Formatted at 2026-08-08 10:45:28 on dist-test-slave-twwt"
server_key: "cdbf1cb3867d0a114f0ab2b510e82ad9"
server_key_iv: "ccaac9743b0d0c30bde0876e875b9ec8"
server_key_version: "encryptionkey@0"
I20260808 10:45:28.178987 15589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:28.183866 15589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:28.184469 15589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:28.184645 15589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:28.185192 15589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:28.186090 15589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:28.186159 15589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:28.186225 15589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:28.186276 15589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:28.195091 15589 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:39639
I20260808 10:45:28.195152 15802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:39639 every 8 connection(s)
I20260808 10:45:28.196013 15589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:45:28.199795 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 15589
I20260808 10:45:28.199957 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:45:28.200256 32694 external_mini_cluster.cc:1428] Setting key e7953699ac57203b6520989f3ac200f3
I20260808 10:45:28.202452 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44263
--builtin_ntp_servers=127.31.237.148:44403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 10:45:28.202775 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:28.197912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:51793 (local address 127.31.237.190:44263)
0808 10:45:28.198120 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:28.198125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:28.199346 (+ 1221us) server_negotiation.cc:407] Connection header received
0808 10:45:28.200831 (+ 1485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:28.200835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:28.200880 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:28.200978 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:28.202206 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:28.202212 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:28.202218 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:45:28.202235 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:28.202261 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:45:28.202275 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:28.202383 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:28.202517 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 10:45:28.202567 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 10:45:28.203578 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:28.198440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:44263 (local address 127.31.237.129:51793)
0808 10:45:28.199191 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:28.199232 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:28.200563 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:28.201169 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:28.201177 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:28.201282 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:28.201928 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:45:28.201930 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:28.201969 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:45:28.202037 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:28.202369 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:28.202388 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:28.202392 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:45:28.202520 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 10:45:28.202809 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":197,"threads_started":1}
I20260808 10:45:28.204792 15805 heartbeater.cc:347] Connected to a master server at 127.31.237.190:44263
I20260808 10:45:28.205020 15805 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:28.205425 15805 heartbeater.cc:511] Master 127.31.237.190:44263 requested a full tablet report, sending...
I20260808 10:45:28.206964 15495 ts_manager.cc:194] Registered new tserver with Master: 1ce587b874804707969828d739e85970 (127.31.237.129:39639)
W20260808 10:45:28.317425 15812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:28.317675 15812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:28.317713 15812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:28.322165 15812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:28.322218 15812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:28.322309 15812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:45:28.327402 15812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:28.328461 15812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:28.329900 15812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:28.339762 15825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:28.339799 15812 server_base.cc:1034] running on GCE node
W20260808 10:45:28.339749 15822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:28.339749 15831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:28.340512 15812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:28.341151 15812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:28.342335 15812 hybrid_clock.cc:648] HybridClock initialized: now 1786185928342305 us; error 40 us; skew 500 ppm
I20260808 10:45:28.344204 15812 webserver.cc:460] Webserver started at http://127.31.237.130:36655/ using document root <none> and password file <none>
I20260808 10:45:28.344827 15812 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:28.344949 15812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:28.345168 15812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:28.347434 15812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "cd4398ef8f2b463fb4aee16cbb73d4bb"
format_stamp: "Formatted at 2026-08-08 10:45:28 on dist-test-slave-twwt"
server_key: "ddc0c50123b6c2aacbc847797968f9bc"
server_key_iv: "65517f73136e60a5786fef46d8c86605"
server_key_version: "encryptionkey@0"
I20260808 10:45:28.347962 15812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "cd4398ef8f2b463fb4aee16cbb73d4bb"
format_stamp: "Formatted at 2026-08-08 10:45:28 on dist-test-slave-twwt"
server_key: "ddc0c50123b6c2aacbc847797968f9bc"
server_key_iv: "65517f73136e60a5786fef46d8c86605"
server_key_version: "encryptionkey@0"
I20260808 10:45:28.351948 15812 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:45:28.354578 15841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:28.355847 15812 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:45:28.355984 15812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "cd4398ef8f2b463fb4aee16cbb73d4bb"
format_stamp: "Formatted at 2026-08-08 10:45:28 on dist-test-slave-twwt"
server_key: "ddc0c50123b6c2aacbc847797968f9bc"
server_key_iv: "65517f73136e60a5786fef46d8c86605"
server_key_version: "encryptionkey@0"
I20260808 10:45:28.356107 15812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:28.360870 15812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:28.361500 15812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:28.361678 15812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:28.362232 15812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:28.363226 15812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:28.363296 15812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:28.363370 15812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:28.363409 15812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:28.372733 15812 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:42667
I20260808 10:45:28.372761 16025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:42667 every 8 connection(s)
I20260808 10:45:28.373668 15812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:45:28.379887 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 15812
I20260808 10:45:28.380035 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:45:28.380180 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:28.375721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:48359 (local address 127.31.237.190:44263)
0808 10:45:28.375873 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:28.375878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:28.376999 (+ 1121us) server_negotiation.cc:407] Connection header received
0808 10:45:28.378341 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:28.378345 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:28.378392 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:28.378478 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:28.379573 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:28.379577 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:28.379580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:45:28.379592 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:28.379647 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:45:28.379658 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:28.379756 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:28.379976 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 10:45:28.380032 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:45:28.380434 32694 external_mini_cluster.cc:1428] Setting key f7eaef2b099ce880e1e26d535342d396
I20260808 10:45:28.381462 16031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:28.376158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:44263 (local address 127.31.237.130:48359)
0808 10:45:28.376849 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:28.376889 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:28.378120 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:28.378663 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:28.378671 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:28.378759 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:28.379295 (+ 536us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:45:28.379297 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:28.379336 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:45:28.379422 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:28.379765 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:28.379807 (+ 42us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:28.379813 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:45:28.380007 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 10:45:28.380290 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":126,"threads_started":1}
I20260808 10:45:28.382833 16027 heartbeater.cc:347] Connected to a master server at 127.31.237.190:44263
I20260808 10:45:28.383013 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44263
--builtin_ntp_servers=127.31.237.148:44403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 10:45:28.383106 16027 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:28.383608 16027 heartbeater.cc:511] Master 127.31.237.190:44263 requested a full tablet report, sending...
I20260808 10:45:28.384791 15495 ts_manager.cc:194] Registered new tserver with Master: cd4398ef8f2b463fb4aee16cbb73d4bb (127.31.237.130:42667)
W20260808 10:45:28.500975 16032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:28.501233 16032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:28.501273 16032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:28.505237 16032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:28.505285 16032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:28.505343 16032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:45:28.509917 16032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:44263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:28.511008 16032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:28.512354 16032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:28.522522 16042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:28.522541 16032 server_base.cc:1034] running on GCE node
W20260808 10:45:28.522539 16040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:28.522570 16039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:28.523156 16032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:28.523808 16032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:28.524994 16032 hybrid_clock.cc:648] HybridClock initialized: now 1786185928524973 us; error 29 us; skew 500 ppm
I20260808 10:45:28.526958 16032 webserver.cc:460] Webserver started at http://127.31.237.131:39075/ using document root <none> and password file <none>
I20260808 10:45:28.527550 16032 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:28.527642 16032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:28.527884 16032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:28.529950 16032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "ff73e7963655475c8faa13dd3a96de96"
format_stamp: "Formatted at 2026-08-08 10:45:28 on dist-test-slave-twwt"
server_key: "4e0d54916a95842f156fdaf84a96c019"
server_key_iv: "652444217c87f4dba01f237ac7a93548"
server_key_version: "encryptionkey@0"
I20260808 10:45:28.530575 16032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "ff73e7963655475c8faa13dd3a96de96"
format_stamp: "Formatted at 2026-08-08 10:45:28 on dist-test-slave-twwt"
server_key: "4e0d54916a95842f156fdaf84a96c019"
server_key_iv: "652444217c87f4dba01f237ac7a93548"
server_key_version: "encryptionkey@0"
I20260808 10:45:28.534281 16032 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:45:28.536940 16053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:28.538189 16032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:45:28.538344 16032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "ff73e7963655475c8faa13dd3a96de96"
format_stamp: "Formatted at 2026-08-08 10:45:28 on dist-test-slave-twwt"
server_key: "4e0d54916a95842f156fdaf84a96c019"
server_key_iv: "652444217c87f4dba01f237ac7a93548"
server_key_version: "encryptionkey@0"
I20260808 10:45:28.538460 16032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:28.543102 16032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:28.543689 16032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:28.543871 16032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:28.544421 16032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:28.545372 16032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:28.545441 16032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:28.545507 16032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:28.545553 16032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:28.554474 16032 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:46079
I20260808 10:45:28.554509 16233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:46079 every 8 connection(s)
I20260808 10:45:28.555394 16032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:45:28.560737 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 16032
I20260808 10:45:28.560856 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:45:28.561199 32694 external_mini_cluster.cc:1428] Setting key 64277ebb40bfae053f45f0d260bcea33
I20260808 10:45:28.561609 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:28.557132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:59583 (local address 127.31.237.190:44263)
0808 10:45:28.557263 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:28.557266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:28.558283 (+ 1017us) server_negotiation.cc:407] Connection header received
0808 10:45:28.559725 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:28.559728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:28.559767 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:28.559843 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:28.561119 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:28.561122 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:28.561124 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:45:28.561138 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:28.561160 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:45:28.561169 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:28.561253 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:28.561428 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 10:45:28.561472 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:45:28.562772 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:28.557474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:44263 (local address 127.31.237.131:59583)
0808 10:45:28.558136 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:28.558177 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:28.559517 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:28.560025 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:28.560032 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:28.560147 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:28.560741 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:45:28.560743 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:28.560806 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:45:28.560942 (+ 136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:28.561263 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:28.561272 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:28.561278 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:45:28.561472 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 10:45:28.561739 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":111,"threads_started":1}
I20260808 10:45:28.563864 16235 heartbeater.cc:347] Connected to a master server at 127.31.237.190:44263
I20260808 10:45:28.564073 16235 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:28.564445 16235 heartbeater.cc:511] Master 127.31.237.190:44263 requested a full tablet report, sending...
I20260808 10:45:28.565450 15493 ts_manager.cc:194] Registered new tserver with Master: ff73e7963655475c8faa13dd3a96de96 (127.31.237.131:46079)
I20260808 10:45:28.575424 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:45:28.578387 15562 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:45:28.577399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50698 (local address 127.31.237.190:44263)
0808 10:45:28.577557 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:28.577562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:28.577715 (+ 153us) server_negotiation.cc:407] Connection header received
0808 10:45:28.577893 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:28.577896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:28.577932 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:28.578011 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:28.578249 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50698: BlockingRecv error: recv got EOF from 127.0.0.1:50698 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:45:28.578593 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 15589
I20260808 10:45:28.589615 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 15812
I20260808 10:45:28.604002 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 16032
I20260808 10:45:28.614781 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 15443
2026-08-08T10:45:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (867 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:45:28 dist-test-slave-twwt krb5kdc[16248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:45:28 dist-test-slave-twwt krb5kdc[16248](info): set up 2 sockets
Aug 08 10:45:28 dist-test-slave-twwt krb5kdc[16248](info): commencing operation
krb5kdc: starting...
W20260808 10:45:29.865337 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:45:29 dist-test-slave-twwt krb5kdc[16248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:45:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:45:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:45:30.002014 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:39275
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:38353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:39275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:45:30.118767 16268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:30.119076 16268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:30.119118 16268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:30.123139 16268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:45:30.123241 16268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:30.123278 16268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:30.123303 16268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:45:30.123323 16268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:45:30.128773 16268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:38353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:39275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:39275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:30.129827 16268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:30.130901 16268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:30.139647 16273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:30.139653 16276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:30.139647 16274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:30.140004 16268 server_base.cc:1034] running on GCE node
I20260808 10:45:30.140559 16268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:30.141865 16268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:30.143034 16268 hybrid_clock.cc:648] HybridClock initialized: now 1786185930143007 us; error 45 us; skew 500 ppm
Aug 08 10:45:30 dist-test-slave-twwt krb5kdc[16248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185930, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:30.145843 16268 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:45:30.146857 16268 webserver.cc:460] Webserver started at http://127.31.237.190:40885/ using document root <none> and password file <none>
I20260808 10:45:30.147449 16268 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:30.147541 16268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:30.147781 16268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:30.150024 16268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "9720eb493b4d4fd3bdcc7b94b8506bba"
format_stamp: "Formatted at 2026-08-08 10:45:30 on dist-test-slave-twwt"
server_key: "889f6c31a5afb050a97182933d16cd6d"
server_key_iv: "466a759966ea8920a5bec44ada909fbb"
server_key_version: "encryptionkey@0"
I20260808 10:45:30.150667 16268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "9720eb493b4d4fd3bdcc7b94b8506bba"
format_stamp: "Formatted at 2026-08-08 10:45:30 on dist-test-slave-twwt"
server_key: "889f6c31a5afb050a97182933d16cd6d"
server_key_iv: "466a759966ea8920a5bec44ada909fbb"
server_key_version: "encryptionkey@0"
I20260808 10:45:30.154572 16268 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 10:45:30.157076 16285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:30.158316 16268 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:45:30.158563 16268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "9720eb493b4d4fd3bdcc7b94b8506bba"
format_stamp: "Formatted at 2026-08-08 10:45:30 on dist-test-slave-twwt"
server_key: "889f6c31a5afb050a97182933d16cd6d"
server_key_iv: "466a759966ea8920a5bec44ada909fbb"
server_key_version: "encryptionkey@0"
I20260808 10:45:30.158721 16268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:30.176397 16268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:30.179662 16268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:30.179883 16268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:30.187177 16268 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:39275
I20260808 10:45:30.187471 16368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:39275 every 8 connection(s)
I20260808 10:45:30.188242 16268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:45:30.189421 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 16268
I20260808 10:45:30.189567 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:45:30.189872 32694 external_mini_cluster.cc:1428] Setting key a2b5461b8f859a7a835ba8b9173ce747
I20260808 10:45:30.191725 16369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:30.197716 16369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba: Bootstrap starting.
Aug 08 10:45:30 dist-test-slave-twwt krb5kdc[16248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:45:30.200587 16369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:30.201377 16369 log.cc:824] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:30.203770 16369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba: No bootstrap required, opened a new log
I20260808 10:45:30.205274 16374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:30.191320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39600 (local address 127.31.237.190:39275)
0808 10:45:30.191750 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:30.191772 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:30.192015 (+ 243us) server_negotiation.cc:407] Connection header received
0808 10:45:30.192812 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:30.192829 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:30.193172 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:30.193516 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:30.194352 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:30.195437 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:30.196288 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:30.196759 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:30.196962 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:30.199959 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:30.199964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:30.199977 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:30.200004 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:30.201991 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:30.202550 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:30.202556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:30.202560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:30.202728 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:30.203266 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:30.203269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:30.203271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:30.203681 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:30.203929 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:30.204198 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 10:45:30.204517 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":133,"threads_started":1}
I20260808 10:45:30.208149 16369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9720eb493b4d4fd3bdcc7b94b8506bba" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39275 } }
I20260808 10:45:30.208384 16369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:30.208464 16369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9720eb493b4d4fd3bdcc7b94b8506bba, State: Initialized, Role: FOLLOWER
I20260808 10:45:30.208891 16369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9720eb493b4d4fd3bdcc7b94b8506bba" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39275 } }
I20260808 10:45:30.209039 16369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:45:30.209110 16369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:45:30.209229 16369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:30.210172 16369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9720eb493b4d4fd3bdcc7b94b8506bba" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39275 } }
I20260808 10:45:30.210532 16369 leader_election.cc:302] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9720eb493b4d4fd3bdcc7b94b8506bba; no voters:
I20260808 10:45:30.210888 16369 leader_election.cc:288] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:45:30.211014 16379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:45:30.211239 16379 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [term 1 LEADER]: Becoming Leader. State: Replica: 9720eb493b4d4fd3bdcc7b94b8506bba, State: Running, Role: LEADER
I20260808 10:45:30.211647 16379 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9720eb493b4d4fd3bdcc7b94b8506bba" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39275 } }
I20260808 10:45:30.212261 16369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:45:30.213506 16381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9720eb493b4d4fd3bdcc7b94b8506bba. Latest consensus state: current_term: 1 leader_uuid: "9720eb493b4d4fd3bdcc7b94b8506bba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9720eb493b4d4fd3bdcc7b94b8506bba" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39275 } } }
I20260808 10:45:30.213644 16381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:30.214030 16380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9720eb493b4d4fd3bdcc7b94b8506bba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9720eb493b4d4fd3bdcc7b94b8506bba" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39275 } } }
I20260808 10:45:30.214155 16380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba [sys.catalog]: This master's current role is: LEADER
I20260808 10:45:30.214715 16388 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:45:30.217479 16388 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:45:30.222934 16388 catalog_manager.cc:1347] Generated new cluster ID: 08b0324c12314c97acd0e094f9631f34
I20260808 10:45:30.223011 16388 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:45:30.245347 16388 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:45:30.246659 16388 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:45:30.254346 16388 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba: Generated new TSK 0
I20260808 10:45:30.255213 16388 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:45:30.290267 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39275
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:38353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:45:30.414585 16411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:30.414908 16411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:30.414958 16411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:30.419548 16411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:30.419615 16411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:30.419672 16411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:45:30.425076 16411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:38353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:30.426141 16411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:30.427215 16411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:30.437471 16417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:30.437561 16411 server_base.cc:1034] running on GCE node
W20260808 10:45:30.437482 16423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:30.437789 16420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:30.438200 16411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:30.438896 16411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:30.440075 16411 hybrid_clock.cc:648] HybridClock initialized: now 1786185930440058 us; error 28 us; skew 500 ppm
Aug 08 10:45:30 dist-test-slave-twwt krb5kdc[16248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185930, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:30.442907 16411 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:45:30.443743 16411 webserver.cc:460] Webserver started at http://127.31.237.129:41219/ using document root <none> and password file <none>
I20260808 10:45:30.444311 16411 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:30.444401 16411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:30.444633 16411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:30.446712 16411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "445c2b189c6442f2a7bf5b318f28a357"
format_stamp: "Formatted at 2026-08-08 10:45:30 on dist-test-slave-twwt"
server_key: "18879ad59502853866567abf83e75cee"
server_key_iv: "09235e945f562415f9aba5ded2efe49d"
server_key_version: "encryptionkey@0"
I20260808 10:45:30.447292 16411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "445c2b189c6442f2a7bf5b318f28a357"
format_stamp: "Formatted at 2026-08-08 10:45:30 on dist-test-slave-twwt"
server_key: "18879ad59502853866567abf83e75cee"
server_key_iv: "09235e945f562415f9aba5ded2efe49d"
server_key_version: "encryptionkey@0"
I20260808 10:45:30.451077 16411 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 10:45:30.453532 16433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:30.454748 16411 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:45:30.454891 16411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "445c2b189c6442f2a7bf5b318f28a357"
format_stamp: "Formatted at 2026-08-08 10:45:30 on dist-test-slave-twwt"
server_key: "18879ad59502853866567abf83e75cee"
server_key_iv: "09235e945f562415f9aba5ded2efe49d"
server_key_version: "encryptionkey@0"
I20260808 10:45:30.455003 16411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:30.492250 16411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:30.495738 16411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:30.495951 16411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:30.496574 16411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:30.497430 16411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:30.497499 16411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:30.497571 16411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:30.497612 16411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:30.506723 16411 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:34235
I20260808 10:45:30.506768 16611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:34235 every 8 connection(s)
I20260808 10:45:30.507648 16411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:45:30.508612 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 16411
I20260808 10:45:30.508756 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:45:30.509083 32694 external_mini_cluster.cc:1428] Setting key 32adb0ffbf28af124c7c5095a9cd76c4
Aug 08 10:45:30 dist-test-slave-twwt krb5kdc[16248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185930, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
I20260808 10:45:30.523973 16374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:30.509716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:59223 (local address 127.31.237.190:39275)
0808 10:45:30.509869 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:30.509873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:30.510928 (+ 1055us) server_negotiation.cc:407] Connection header received
0808 10:45:30.512431 (+ 1503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:30.512437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:30.512502 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:30.512602 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:30.514494 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:30.515074 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:30.516947 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:30.517175 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:30.517261 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:30.520198 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:30.520201 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:30.520203 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:30.520227 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:30.521725 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:30.522298 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:30.522302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:30.522304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:30.522388 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:30.522772 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:30.522778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:30.522779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:30.522969 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:30.523079 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:30.523625 (+ 546us) server_negotiation.cc:299] Negotiation successful
0808 10:45:30.523766 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:45:30.524868 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:30.510195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39275 (local address 127.31.237.129:59223)
0808 10:45:30.510753 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:30.510816 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:30.512218 (+ 1402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:30.512852 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:30.512865 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:30.513479 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:30.514325 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:30.514352 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:30.515210 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:30.515216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:30.516806 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:30.516823 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:30.517340 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:30.517348 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:30.517601 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:30.518282 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:30.518306 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:30.520022 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:30.521902 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:30.521908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:30.521923 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:30.522190 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:30.522480 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:30.522484 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:30.522486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:30.522673 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:30.523078 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:30.523085 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:30.523348 (+ 263us) client_negotiation.cc:769] Sending connection context
0808 10:45:30.523566 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 10:45:30.523926 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":150,"threads_started":1}
I20260808 10:45:30.526154 16612 heartbeater.cc:347] Connected to a master server at 127.31.237.190:39275
I20260808 10:45:30.526402 16612 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:30.526958 16612 heartbeater.cc:511] Master 127.31.237.190:39275 requested a full tablet report, sending...
I20260808 10:45:30.528466 16308 ts_manager.cc:194] Registered new tserver with Master: 445c2b189c6442f2a7bf5b318f28a357 (127.31.237.129:34235)
I20260808 10:45:30.529672 16308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:59223
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:45:30.541632 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39275
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:38353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:45:30.664927 16621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:30.665198 16621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:30.665244 16621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:30.669170 16621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:30.669234 16621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:30.669292 16621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:45:30.674569 16621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:38353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:30.675673 16621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:30.676726 16621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:30.687161 16638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:30.687148 16634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:30.687148 16632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:30.687198 16621 server_base.cc:1034] running on GCE node
I20260808 10:45:30.687857 16621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:30.688541 16621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:30.689724 16621 hybrid_clock.cc:648] HybridClock initialized: now 1786185930689688 us; error 47 us; skew 500 ppm
Aug 08 10:45:30 dist-test-slave-twwt krb5kdc[16248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185930, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:30.698256 16621 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:45:30.699324 16621 webserver.cc:460] Webserver started at http://127.31.237.130:32825/ using document root <none> and password file <none>
I20260808 10:45:30.699928 16621 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:30.700018 16621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:30.700254 16621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:30.702477 16621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "d0a50bf017b2466c94530ea977a9571b"
format_stamp: "Formatted at 2026-08-08 10:45:30 on dist-test-slave-twwt"
server_key: "8008d0999846db70b94ccd95e6bf3119"
server_key_iv: "a7f70aa6044fa06baad001cb64de5497"
server_key_version: "encryptionkey@0"
I20260808 10:45:30.703127 16621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "d0a50bf017b2466c94530ea977a9571b"
format_stamp: "Formatted at 2026-08-08 10:45:30 on dist-test-slave-twwt"
server_key: "8008d0999846db70b94ccd95e6bf3119"
server_key_iv: "a7f70aa6044fa06baad001cb64de5497"
server_key_version: "encryptionkey@0"
I20260808 10:45:30.706908 16621 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:45:30.709404 16646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:30.710670 16621 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:45:30.710798 16621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "d0a50bf017b2466c94530ea977a9571b"
format_stamp: "Formatted at 2026-08-08 10:45:30 on dist-test-slave-twwt"
server_key: "8008d0999846db70b94ccd95e6bf3119"
server_key_iv: "a7f70aa6044fa06baad001cb64de5497"
server_key_version: "encryptionkey@0"
I20260808 10:45:30.710906 16621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:30.751341 16621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:30.755187 16621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:30.755415 16621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:30.756110 16621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:30.757053 16621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:30.757128 16621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:30.757203 16621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:30.757244 16621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:30.767666 16621 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:39289
I20260808 10:45:30.767697 16833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:39289 every 8 connection(s)
I20260808 10:45:30.768719 16621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:45:30.770114 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 16621
I20260808 10:45:30.770259 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:45:30.770596 32694 external_mini_cluster.cc:1428] Setting key aa22fab3b26cf15a9366e7bfcc951b33
Aug 08 10:45:30 dist-test-slave-twwt krb5kdc[16248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185930, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
I20260808 10:45:30.786111 16374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:30.771025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:50469 (local address 127.31.237.190:39275)
0808 10:45:30.771193 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:30.771198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:30.772403 (+ 1205us) server_negotiation.cc:407] Connection header received
0808 10:45:30.774716 (+ 2313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:30.774722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:30.774797 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:30.774953 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:30.776296 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:30.776839 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:30.778175 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:30.778417 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:30.778528 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:30.781718 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:30.781722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:30.781725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:30.781761 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:30.783624 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:30.784384 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:30.784388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:30.784390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:30.784484 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:30.784940 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:30.784944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:30.784946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:30.785162 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:30.785265 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:30.785805 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 10:45:30.785941 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 10:45:30.787214 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:30.771527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39275 (local address 127.31.237.130:50469)
0808 10:45:30.772231 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:30.772302 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:30.774363 (+ 2061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:30.775152 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:30.775161 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:30.775570 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:30.776123 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:30.776138 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:30.776980 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:30.776984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:30.778033 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:30.778051 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:30.778580 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:30.778584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:30.778801 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:30.779529 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:30.779556 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:30.781543 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:30.783770 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:30.783777 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:30.783796 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:30.784244 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:30.784587 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:30.784590 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:30.784592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:30.784829 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:30.785279 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:30.785288 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:30.785562 (+ 274us) client_negotiation.cc:769] Sending connection context
0808 10:45:30.785759 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 10:45:30.786126 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":176,"threads_started":1}
I20260808 10:45:30.788681 16834 heartbeater.cc:347] Connected to a master server at 127.31.237.190:39275
I20260808 10:45:30.788945 16834 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:30.789495 16834 heartbeater.cc:511] Master 127.31.237.190:39275 requested a full tablet report, sending...
I20260808 10:45:30.790683 16308 ts_manager.cc:194] Registered new tserver with Master: d0a50bf017b2466c94530ea977a9571b (127.31.237.130:39289)
I20260808 10:45:30.791285 16308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:50469
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:45:30.802026 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39275
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:38353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:45:30.925235 16840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:45:30.925522 16840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:45:30.925595 16840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:45:30.929683 16840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:45:30.929768 16840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:45:30.929858 16840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:45:30.935220 16840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:38353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:39275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:45:30.936288 16840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:45:30.937337 16840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:45:30.946978 16847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:30.946978 16848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:45:30.947017 16852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:45:30.947186 16840 server_base.cc:1034] running on GCE node
I20260808 10:45:30.947767 16840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:45:30.948400 16840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:45:30.949582 16840 hybrid_clock.cc:648] HybridClock initialized: now 1786185930949561 us; error 29 us; skew 500 ppm
Aug 08 10:45:30 dist-test-slave-twwt krb5kdc[16248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185930, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:45:30.952786 16840 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:45:30.953733 16840 webserver.cc:460] Webserver started at http://127.31.237.131:46413/ using document root <none> and password file <none>
I20260808 10:45:30.954324 16840 fs_manager.cc:359] Metadata directory not provided
I20260808 10:45:30.954398 16840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:45:30.954715 16840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:45:30.956905 16840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "a7f860ce3d1b4887883654d2e8d13765"
format_stamp: "Formatted at 2026-08-08 10:45:30 on dist-test-slave-twwt"
server_key: "ef39cc1399e0f144ab46501ca2b90515"
server_key_iv: "3f37fec042290eb1aa2ccbc3d6ab64c4"
server_key_version: "encryptionkey@0"
I20260808 10:45:30.957525 16840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "a7f860ce3d1b4887883654d2e8d13765"
format_stamp: "Formatted at 2026-08-08 10:45:30 on dist-test-slave-twwt"
server_key: "ef39cc1399e0f144ab46501ca2b90515"
server_key_iv: "3f37fec042290eb1aa2ccbc3d6ab64c4"
server_key_version: "encryptionkey@0"
I20260808 10:45:30.961758 16840 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:45:30.964395 16862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:30.965577 16840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:45:30.965698 16840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "a7f860ce3d1b4887883654d2e8d13765"
format_stamp: "Formatted at 2026-08-08 10:45:30 on dist-test-slave-twwt"
server_key: "ef39cc1399e0f144ab46501ca2b90515"
server_key_iv: "3f37fec042290eb1aa2ccbc3d6ab64c4"
server_key_version: "encryptionkey@0"
I20260808 10:45:30.965804 16840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:45:30.980803 16840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:45:30.984189 16840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:45:30.984398 16840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:45:30.985023 16840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:45:30.985870 16840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:45:30.985918 16840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:30.985985 16840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:45:30.986021 16840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:45:30.999630 16840 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:34385
I20260808 10:45:30.999963 17023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:34385 every 8 connection(s)
I20260808 10:45:31.000995 16840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:45:31.009626 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 16840
I20260808 10:45:31.009763 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:45:31.010095 32694 external_mini_cluster.cc:1428] Setting key c513e639b3cadb6e816c7a3688932f3f
Aug 08 10:45:31 dist-test-slave-twwt krb5kdc[16248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185930, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:45:31.020206 16374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.003127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:45153 (local address 127.31.237.190:39275)
0808 10:45:31.003282 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:31.003287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:31.004470 (+ 1183us) server_negotiation.cc:407] Connection header received
0808 10:45:31.006478 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:31.006484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:31.006552 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:31.006623 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:31.008458 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.009230 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.010926 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.011158 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.011278 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:31.014362 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:31.014365 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:31.014368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:31.014402 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:31.017905 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:31.018506 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:31.018510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:31.018512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:31.018635 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:31.018993 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:31.018998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:31.019000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:31.019240 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:31.019370 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:31.019904 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 10:45:31.020038 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:45:31.021210 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.003655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:39275 (local address 127.31.237.131:45153)
0808 10:45:31.004289 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:31.004346 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:31.006223 (+ 1877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:31.007039 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:31.007048 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:31.007477 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:45:31.008245 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:31.008267 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.009379 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.009384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:31.010775 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:31.010786 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.011317 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.011321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:31.011546 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:31.012302 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:45:31.012329 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:45:31.014147 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:45:31.018063 (+ 3916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:31.018069 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:31.018084 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:31.018394 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:31.018716 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:45:31.018720 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:45:31.018722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:45:31.018892 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:45:31.019368 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:45:31.019375 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:45:31.019643 (+ 268us) client_negotiation.cc:769] Sending connection context
0808 10:45:31.019859 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 10:45:31.020206 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":163,"threads_started":1}
I20260808 10:45:31.022572 17027 heartbeater.cc:347] Connected to a master server at 127.31.237.190:39275
I20260808 10:45:31.022874 17027 heartbeater.cc:464] Registering TS with master...
I20260808 10:45:31.023506 17027 heartbeater.cc:511] Master 127.31.237.190:39275 requested a full tablet report, sending...
I20260808 10:45:31.024832 16307 ts_manager.cc:194] Registered new tserver with Master: a7f860ce3d1b4887883654d2e8d13765 (127.31.237.131:34385)
I20260808 10:45:31.025473 16307 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:45153
I20260808 10:45:31.036684 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:45:31.046887 16374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.039132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39616 (local address 127.31.237.190:39275)
0808 10:45:31.039272 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:31.039275 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:31.039518 (+ 243us) server_negotiation.cc:407] Connection header received
0808 10:45:31.039683 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:31.039687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:31.039748 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:31.039830 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:45:31.040583 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.041114 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.042000 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.042236 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.042344 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:31.043842 (+ 1498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:45:31.043845 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:45:31.043847 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:45:31.043874 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:45:31.045084 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:31.045541 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:31.045544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:31.045546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:31.045633 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:45:31.045913 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:45:31.045916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:45:31.045918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:45:31.046161 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:45:31.046308 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:31.046560 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 10:45:31.046709 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:45:31.049981 16307 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:45:31.053422 16307 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:45:31.068620 17047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.063340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45162 (local address 127.31.237.131:34385)
0808 10:45:31.063751 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:31.063765 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:31.063941 (+ 176us) server_negotiation.cc:407] Connection header received
0808 10:45:31.064064 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:31.064071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:31.064306 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:31.064451 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:31.064681 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.065436 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.067122 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.067902 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.068054 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:31.068196 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:31.068368 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 10:45:31.068459 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":208,"threads_started":1}
I20260808 10:45:31.068612 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.062649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:34385 (local address 127.0.0.1:45162)
0808 10:45:31.063842 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:31.063856 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:31.063950 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:31.064451 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:31.064454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:31.064470 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:31.064582 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:31.064588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.065570 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.065577 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:31.066978 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:31.066990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.068042 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.068046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:31.068162 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:31.068201 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 10:45:31.068332 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 10:45:31.068456 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1134,"thread_start_us":163,"threads_started":1}
I20260808 10:45:31.069252 17045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.062472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:34235 (local address 127.0.0.1:37486)
0808 10:45:31.063324 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:31.063375 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:31.063524 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:31.064232 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:31.064241 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:31.064268 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:31.064451 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:31.064458 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.066389 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.066393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:31.067521 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:31.067532 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.068795 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.068801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:31.068958 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:31.068992 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 10:45:31.069056 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 10:45:31.069125 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":23,"thread_start_us":81,"threads_started":1}
I20260808 10:45:31.069715 17044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.062609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37486 (local address 127.31.237.129:34235)
0808 10:45:31.063047 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:31.063055 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:31.063677 (+ 622us) server_negotiation.cc:407] Connection header received
0808 10:45:31.063769 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:31.063775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:31.064021 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:31.064590 (+ 569us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:31.064748 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.065454 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.067653 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.068479 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.068611 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:31.068727 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:31.069453 (+ 726us) server_negotiation.cc:299] Negotiation successful
0808 10:45:31.069553 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":91,"threads_started":1}
I20260808 10:45:31.070655 17043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.062890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:39289 (local address 127.0.0.1:52658)
0808 10:45:31.064123 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:31.064136 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:31.064207 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:31.066222 (+ 2015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:31.066225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:31.066243 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:31.066398 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:31.066407 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.067718 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.067722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:31.068937 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:31.068947 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.070216 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.070219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:31.070320 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:31.070346 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 10:45:31.070426 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 10:45:31.070492 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1160,"mutex_wait_us":23,"thread_start_us":94,"threads_started":1}
I20260808 10:45:31.072597 16510 tablet_service.cc:1467] Processing CreateTablet for tablet df46b26ff4e847cb94ff2285b8a8234e (DEFAULT_TABLE table=test-table [id=ff5de76a951b4b6aadc1daf3da9d7ddc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:31.072455 16932 tablet_service.cc:1467] Processing CreateTablet for tablet df46b26ff4e847cb94ff2285b8a8234e (DEFAULT_TABLE table=test-table [id=ff5de76a951b4b6aadc1daf3da9d7ddc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:31.073674 16932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df46b26ff4e847cb94ff2285b8a8234e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:31.073673 16510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df46b26ff4e847cb94ff2285b8a8234e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:31.074106 17049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.063365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52658 (local address 127.31.237.130:39289)
0808 10:45:31.065756 (+ 2391us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:31.065767 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:31.065789 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:45:31.065875 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:31.065880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:31.066053 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:31.066209 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:31.066566 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.067545 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.069062 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.070082 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.070206 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:31.070323 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:31.070464 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 10:45:31.070551 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2286,"thread_start_us":165,"threads_started":1}
I20260808 10:45:31.076680 16717 tablet_service.cc:1467] Processing CreateTablet for tablet df46b26ff4e847cb94ff2285b8a8234e (DEFAULT_TABLE table=test-table [id=ff5de76a951b4b6aadc1daf3da9d7ddc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:45:31.077602 16717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df46b26ff4e847cb94ff2285b8a8234e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:45:31.081446 17051 tablet_bootstrap.cc:492] T df46b26ff4e847cb94ff2285b8a8234e P 445c2b189c6442f2a7bf5b318f28a357: Bootstrap starting.
I20260808 10:45:31.081900 17052 tablet_bootstrap.cc:492] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765: Bootstrap starting.
I20260808 10:45:31.084089 17054 tablet_bootstrap.cc:492] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b: Bootstrap starting.
I20260808 10:45:31.085620 17052 tablet_bootstrap.cc:654] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:31.085795 17051 tablet_bootstrap.cc:654] T df46b26ff4e847cb94ff2285b8a8234e P 445c2b189c6442f2a7bf5b318f28a357: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:31.086433 17052 log.cc:824] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:31.086745 17051 log.cc:824] T df46b26ff4e847cb94ff2285b8a8234e P 445c2b189c6442f2a7bf5b318f28a357: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:31.086815 17054 tablet_bootstrap.cc:654] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b: Neither blocks nor log segments found. Creating new log.
I20260808 10:45:31.087605 17054 log.cc:824] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:45:31.088580 17052 tablet_bootstrap.cc:492] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765: No bootstrap required, opened a new log
I20260808 10:45:31.088788 17052 ts_tablet_manager.cc:1397] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 10:45:31.088827 17051 tablet_bootstrap.cc:492] T df46b26ff4e847cb94ff2285b8a8234e P 445c2b189c6442f2a7bf5b318f28a357: No bootstrap required, opened a new log
I20260808 10:45:31.089063 17051 ts_tablet_manager.cc:1397] T df46b26ff4e847cb94ff2285b8a8234e P 445c2b189c6442f2a7bf5b318f28a357: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260808 10:45:31.089635 17054 tablet_bootstrap.cc:492] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b: No bootstrap required, opened a new log
I20260808 10:45:31.089820 17054 ts_tablet_manager.cc:1397] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:45:31.092238 17052 raft_consensus.cc:348] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445c2b189c6442f2a7bf5b318f28a357" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34235 } } peers { permanent_uuid: "a7f860ce3d1b4887883654d2e8d13765" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34385 } } peers { permanent_uuid: "d0a50bf017b2466c94530ea977a9571b" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 39289 } }
I20260808 10:45:31.092546 17052 raft_consensus.cc:374] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:31.092625 17052 raft_consensus.cc:729] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7f860ce3d1b4887883654d2e8d13765, State: Initialized, Role: FOLLOWER
I20260808 10:45:31.093101 17052 consensus_queue.cc:260] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445c2b189c6442f2a7bf5b318f28a357" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34235 } } peers { permanent_uuid: "a7f860ce3d1b4887883654d2e8d13765" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34385 } } peers { permanent_uuid: "d0a50bf017b2466c94530ea977a9571b" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 39289 } }
I20260808 10:45:31.093777 17027 heartbeater.cc:503] Master 127.31.237.190:39275 was elected leader, sending a full tablet report...
I20260808 10:45:31.093626 17054 raft_consensus.cc:348] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445c2b189c6442f2a7bf5b318f28a357" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34235 } } peers { permanent_uuid: "a7f860ce3d1b4887883654d2e8d13765" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34385 } } peers { permanent_uuid: "d0a50bf017b2466c94530ea977a9571b" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 39289 } }
I20260808 10:45:31.093883 17054 raft_consensus.cc:374] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:31.093941 17054 raft_consensus.cc:729] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0a50bf017b2466c94530ea977a9571b, State: Initialized, Role: FOLLOWER
I20260808 10:45:31.094154 17052 ts_tablet_manager.cc:1428] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 10:45:31.094360 17054 consensus_queue.cc:260] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445c2b189c6442f2a7bf5b318f28a357" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34235 } } peers { permanent_uuid: "a7f860ce3d1b4887883654d2e8d13765" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34385 } } peers { permanent_uuid: "d0a50bf017b2466c94530ea977a9571b" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 39289 } }
I20260808 10:45:31.093626 17051 raft_consensus.cc:348] T df46b26ff4e847cb94ff2285b8a8234e P 445c2b189c6442f2a7bf5b318f28a357 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445c2b189c6442f2a7bf5b318f28a357" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34235 } } peers { permanent_uuid: "a7f860ce3d1b4887883654d2e8d13765" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34385 } } peers { permanent_uuid: "d0a50bf017b2466c94530ea977a9571b" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 39289 } }
I20260808 10:45:31.094571 17051 raft_consensus.cc:374] T df46b26ff4e847cb94ff2285b8a8234e P 445c2b189c6442f2a7bf5b318f28a357 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:45:31.094668 17051 raft_consensus.cc:729] T df46b26ff4e847cb94ff2285b8a8234e P 445c2b189c6442f2a7bf5b318f28a357 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445c2b189c6442f2a7bf5b318f28a357, State: Initialized, Role: FOLLOWER
I20260808 10:45:31.095291 16834 heartbeater.cc:503] Master 127.31.237.190:39275 was elected leader, sending a full tablet report...
I20260808 10:45:31.095355 17054 ts_tablet_manager.cc:1428] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b: Time spent starting tablet: real 0.005s user 0.003s sys 0.004s
I20260808 10:45:31.096892 17051 consensus_queue.cc:260] T df46b26ff4e847cb94ff2285b8a8234e P 445c2b189c6442f2a7bf5b318f28a357 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445c2b189c6442f2a7bf5b318f28a357" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34235 } } peers { permanent_uuid: "a7f860ce3d1b4887883654d2e8d13765" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34385 } } peers { permanent_uuid: "d0a50bf017b2466c94530ea977a9571b" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 39289 } }
I20260808 10:45:31.097611 16612 heartbeater.cc:503] Master 127.31.237.190:39275 was elected leader, sending a full tablet report...
I20260808 10:45:31.097844 17051 ts_tablet_manager.cc:1428] T df46b26ff4e847cb94ff2285b8a8234e P 445c2b189c6442f2a7bf5b318f28a357: Time spent starting tablet: real 0.009s user 0.002s sys 0.004s
I20260808 10:45:31.112541 17059 raft_consensus.cc:482] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:45:31.112736 17059 raft_consensus.cc:504] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445c2b189c6442f2a7bf5b318f28a357" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34235 } } peers { permanent_uuid: "a7f860ce3d1b4887883654d2e8d13765" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34385 } } peers { permanent_uuid: "d0a50bf017b2466c94530ea977a9571b" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 39289 } }
I20260808 10:45:31.114125 17059 leader_election.cc:288] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 445c2b189c6442f2a7bf5b318f28a357 (127.31.237.129:34235), d0a50bf017b2466c94530ea977a9571b (127.31.237.130:39289)
I20260808 10:45:31.117830 17044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.114557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:38315 (local address 127.31.237.129:34235)
0808 10:45:31.114744 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:31.114750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:31.114764 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:45:31.114836 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:31.114839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:31.114894 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:31.114985 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:31.115313 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.115844 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.116864 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.117276 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.117405 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:31.117460 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:31.117638 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 10:45:31.117712 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 10:45:31.117837 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.114330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:34235 (local address 127.31.237.131:38315)
0808 10:45:31.114486 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:31.114502 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:31.114595 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:31.114999 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:31.115002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:31.115072 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:31.115196 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:31.115201 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.115967 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.115971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:31.116734 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:31.116743 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.117405 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.117409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:31.117514 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:31.117537 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:45:31.117622 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:45:31.117694 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"spinlock_wait_cycles":19712}
I20260808 10:45:31.118098 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.114452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:39289 (local address 127.31.237.131:54611)
0808 10:45:31.114876 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:31.114891 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:45:31.115027 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:45:31.115259 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:45:31.115262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:45:31.115277 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:45:31.115432 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:31.115438 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.116271 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.116276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:31.117017 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:45:31.117025 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.117669 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.117673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:45:31.117770 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:31.117792 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:45:31.117887 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 10:45:31.117955 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":151,"threads_started":1}
I20260808 10:45:31.118085 17049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.114594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:54611 (local address 127.31.237.130:39289)
0808 10:45:31.114831 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:31.114836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:31.114984 (+ 148us) server_negotiation.cc:407] Connection header received
0808 10:45:31.115118 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:31.115121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:31.115171 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:31.115251 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:45:31.115548 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.116162 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.117142 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.117542 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.117669 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:31.117719 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:31.117903 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 10:45:31.117972 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260808 10:45:31.118633 16542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df46b26ff4e847cb94ff2285b8a8234e" candidate_uuid: "a7f860ce3d1b4887883654d2e8d13765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445c2b189c6442f2a7bf5b318f28a357" is_pre_election: true
I20260808 10:45:31.118902 16757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df46b26ff4e847cb94ff2285b8a8234e" candidate_uuid: "a7f860ce3d1b4887883654d2e8d13765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0a50bf017b2466c94530ea977a9571b" is_pre_election: true
I20260808 10:45:31.119022 16542 raft_consensus.cc:2454] T df46b26ff4e847cb94ff2285b8a8234e P 445c2b189c6442f2a7bf5b318f28a357 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7f860ce3d1b4887883654d2e8d13765 in term 0.
I20260808 10:45:31.119180 16757 raft_consensus.cc:2454] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7f860ce3d1b4887883654d2e8d13765 in term 0.
I20260808 10:45:31.119582 16864 leader_election.cc:302] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 445c2b189c6442f2a7bf5b318f28a357, a7f860ce3d1b4887883654d2e8d13765, d0a50bf017b2466c94530ea977a9571b; no voters:
I20260808 10:45:31.119875 17059 raft_consensus.cc:2781] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:45:31.119957 17059 raft_consensus.cc:482] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:45:31.119989 17059 raft_consensus.cc:3037] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:31.121091 17059 raft_consensus.cc:504] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445c2b189c6442f2a7bf5b318f28a357" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34235 } } peers { permanent_uuid: "a7f860ce3d1b4887883654d2e8d13765" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34385 } } peers { permanent_uuid: "d0a50bf017b2466c94530ea977a9571b" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 39289 } }
I20260808 10:45:31.121490 17059 leader_election.cc:288] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [CANDIDATE]: Term 1 election: Requested vote from peers 445c2b189c6442f2a7bf5b318f28a357 (127.31.237.129:34235), d0a50bf017b2466c94530ea977a9571b (127.31.237.130:39289)
I20260808 10:45:31.121876 16757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df46b26ff4e847cb94ff2285b8a8234e" candidate_uuid: "a7f860ce3d1b4887883654d2e8d13765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0a50bf017b2466c94530ea977a9571b"
I20260808 10:45:31.121979 16542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df46b26ff4e847cb94ff2285b8a8234e" candidate_uuid: "a7f860ce3d1b4887883654d2e8d13765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "445c2b189c6442f2a7bf5b318f28a357"
I20260808 10:45:31.122033 16757 raft_consensus.cc:3037] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:31.122128 16542 raft_consensus.cc:3037] T df46b26ff4e847cb94ff2285b8a8234e P 445c2b189c6442f2a7bf5b318f28a357 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:45:31.123404 16757 raft_consensus.cc:2454] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7f860ce3d1b4887883654d2e8d13765 in term 1.
I20260808 10:45:31.123390 16542 raft_consensus.cc:2454] T df46b26ff4e847cb94ff2285b8a8234e P 445c2b189c6442f2a7bf5b318f28a357 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7f860ce3d1b4887883654d2e8d13765 in term 1.
I20260808 10:45:31.123749 16867 leader_election.cc:302] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7f860ce3d1b4887883654d2e8d13765, d0a50bf017b2466c94530ea977a9571b; no voters:
I20260808 10:45:31.123987 17059 raft_consensus.cc:2781] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:45:31.124260 17059 raft_consensus.cc:686] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [term 1 LEADER]: Becoming Leader. State: Replica: a7f860ce3d1b4887883654d2e8d13765, State: Running, Role: LEADER
I20260808 10:45:31.124608 17059 consensus_queue.cc:237] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445c2b189c6442f2a7bf5b318f28a357" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34235 } } peers { permanent_uuid: "a7f860ce3d1b4887883654d2e8d13765" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34385 } } peers { permanent_uuid: "d0a50bf017b2466c94530ea977a9571b" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 39289 } }
I20260808 10:45:31.127861 16308 catalog_manager.cc:5495] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7f860ce3d1b4887883654d2e8d13765 (127.31.237.131). New cstate: current_term: 1 leader_uuid: "a7f860ce3d1b4887883654d2e8d13765" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "445c2b189c6442f2a7bf5b318f28a357" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7f860ce3d1b4887883654d2e8d13765" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0a50bf017b2466c94530ea977a9571b" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 39289 } health_report { overall_health: UNKNOWN } } }
I20260808 10:45:31.147887 17047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.144496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45174 (local address 127.31.237.131:34385)
0808 10:45:31.144622 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:45:31.144626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:45:31.144638 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:45:31.144712 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:45:31.144714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:45:31.144770 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:45:31.144867 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:45:31.145103 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.145613 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.146441 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:45:31.146709 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:45:31.146796 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:45:31.147111 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:45:31.147528 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:45:31.147621 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:45:31.147685 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 10:45:31.147736 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:45:31.154095 16542 raft_consensus.cc:1261] T df46b26ff4e847cb94ff2285b8a8234e P 445c2b189c6442f2a7bf5b318f28a357 [term 1 FOLLOWER]: Refusing update from remote peer a7f860ce3d1b4887883654d2e8d13765: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:31.154111 16757 raft_consensus.cc:1261] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b [term 1 FOLLOWER]: Refusing update from remote peer a7f860ce3d1b4887883654d2e8d13765: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:45:31.154841 17059 consensus_queue.cc:1035] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [LEADER]: Connected to new peer: Peer: permanent_uuid: "445c2b189c6442f2a7bf5b318f28a357" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:31.155041 17067 consensus_queue.cc:1035] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0a50bf017b2466c94530ea977a9571b" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 39289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:45:31.162242 17071 mvcc.cc:205] Tried to move back new op lower bound from 7316217573999525888 to 7316217573889622016. Current Snapshot: MvccSnapshot[applied={T|T < 7316217573999525888}]
I20260808 10:45:31.163292 17072 mvcc.cc:205] Tried to move back new op lower bound from 7316217573999525888 to 7316217573889622016. Current Snapshot: MvccSnapshot[applied={T|T < 7316217573999525888}]
I20260808 10:45:31.164597 17073 mvcc.cc:205] Tried to move back new op lower bound from 7316217573999525888 to 7316217573889622016. Current Snapshot: MvccSnapshot[applied={T|T < 7316217573999525888}]
I20260808 10:45:31.174220 16308 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:45:31.174392 16308 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:45:31.176998 16308 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9720eb493b4d4fd3bdcc7b94b8506bba: Sending DeleteTablet for 3 replicas of tablet df46b26ff4e847cb94ff2285b8a8234e
I20260808 10:45:31.177804 16932 tablet_service.cc:1514] Processing DeleteTablet for tablet df46b26ff4e847cb94ff2285b8a8234e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:31 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:45162
I20260808 10:45:31.177850 16510 tablet_service.cc:1514] Processing DeleteTablet for tablet df46b26ff4e847cb94ff2285b8a8234e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:31 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:37486
I20260808 10:45:31.178323 17088 tablet_replica.cc:330] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765: stopping tablet replica
I20260808 10:45:31.178700 17088 raft_consensus.cc:2229] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:45:31.178720 16717 tablet_service.cc:1514] Processing DeleteTablet for tablet df46b26ff4e847cb94ff2285b8a8234e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:45:31 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:52658
I20260808 10:45:31.179173 17088 raft_consensus.cc:2258] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:31.179230 17090 tablet_replica.cc:330] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b: stopping tablet replica
I20260808 10:45:31.179481 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 16411
I20260808 10:45:31.179584 17090 raft_consensus.cc:2229] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:45:31.179922 17090 raft_consensus.cc:2258] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:45:31.183604 17090 ts_tablet_manager.cc:1905] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:31.186641 17090 ts_tablet_manager.cc:1918] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:45:31.186758 17090 log.cc:1196] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/df46b26ff4e847cb94ff2285b8a8234e
I20260808 10:45:31.187088 17090 ts_tablet_manager.cc:1939] T df46b26ff4e847cb94ff2285b8a8234e P d0a50bf017b2466c94530ea977a9571b: Deleting consensus metadata
I20260808 10:45:31.188416 16290 catalog_manager.cc:4841] TS d0a50bf017b2466c94530ea977a9571b (127.31.237.130:39289): tablet df46b26ff4e847cb94ff2285b8a8234e (table test-table [id=ff5de76a951b4b6aadc1daf3da9d7ddc]) successfully deleted
I20260808 10:45:31.188557 17088 ts_tablet_manager.cc:1905] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:45:31.190740 17088 ts_tablet_manager.cc:1918] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:45:31.190856 17088 log.cc:1196] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/df46b26ff4e847cb94ff2285b8a8234e
I20260808 10:45:31.191331 17088 ts_tablet_manager.cc:1939] T df46b26ff4e847cb94ff2285b8a8234e P a7f860ce3d1b4887883654d2e8d13765: Deleting consensus metadata
I20260808 10:45:31.193012 16286 catalog_manager.cc:4841] TS a7f860ce3d1b4887883654d2e8d13765 (127.31.237.131:34385): tablet df46b26ff4e847cb94ff2285b8a8234e (table test-table [id=ff5de76a951b4b6aadc1daf3da9d7ddc]) successfully deleted
W20260808 10:45:31.201114 16287 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:34235 (error 108)
I20260808 10:45:31.201849 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 16621
I20260808 10:45:31.210778 17043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:45:31.210297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:34235 (local address 127.0.0.1:37494)
0808 10:45:31.210546 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 10:45:31.210671 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:34235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260808 10:45:31.211156 16287 catalog_manager.cc:4584] TS 445c2b189c6442f2a7bf5b318f28a357 (127.31.237.129:34235): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df46b26ff4e847cb94ff2285b8a8234e: Network error: Client connection negotiation failed: client connection to 127.31.237.129:34235: connect: Connection refused (error 111)
I20260808 10:45:31.218868 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 16840
I20260808 10:45:31.231794 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 16268
2026-08-08T10:45:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2621 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:45:31 dist-test-slave-twwt krb5kdc[17094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:45:31 dist-test-slave-twwt krb5kdc[17094](info): set up 2 sockets
Aug 08 10:45:31 dist-test-slave-twwt krb5kdc[17094](info): commencing operation
krb5kdc: starting...
W20260808 10:45:32.494503 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:45:32 dist-test-slave-twwt krb5kdc[17094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:45:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:45:32Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.237.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.237.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/rangeradmin_127.31.237.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/rangeradmin_127.31.237.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.237.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.237.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/rangerlookup_127.31.237.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/rangerlookup_127.31.237.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.237.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.237.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/HTTP_127.31.237.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/HTTP_127.31.237.148@KRBTEST.COM.keytab.
I20260808 10:45:32.685937 32694 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDI8wVd/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/postgres -l logfile start
2026-08-08 10:45:35.320 UTC [17139] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 10:45:35.320 UTC [17139] LOG: listening on IPv4 address "127.31.237.148", port 53891
2026-08-08 10:45:35.324 UTC [17139] LOG: listening on Unix socket "/tmp/.s.PGSQL.53891"
2026-08-08 10:45:35.347 UTC [17142] LOG: database system was shut down at 2026-08-08 10:45:33 UTC
2026-08-08 10:45:35.352 UTC [17139] LOG: database system is ready to accept connections
I20260808 10:45:36.510201 32694 mini_postgres.cc:96] Postgres bound to 53891
2026-08-08 10:45:36.518 UTC [17154] FATAL: database "slave" does not exist
127.31.237.148:53891 - accepting connections
I20260808 10:45:36.518760 32694 mini_ranger.cc:161] Starting Ranger...
I20260808 10:45:36.535046 32694 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 10:45:37.047010 32694 mini_ranger.cc:87] Created ranger Postgres database
I20260808 10:45:37.047117 32694 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-admin
2026-08-08 10:45:37,342 [I] DB FLAVOR :POSTGRES
2026-08-08 10:45:37,343 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:45:37,343 [I] Checking connection..
2026-08-08 10:45:37,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 10:45:37,615 [I] Checking connection passed.
2026-08-08 10:45:37,615 [I] --------- Verifying version history table ---------
2026-08-08 10:45:37,616 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:45:37,862 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:45:37,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:45:38,136 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:45:38,137 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 10:45:38,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 10:45:38.381 UTC [17244] WARNING: there is no transaction in progress
2026-08-08 10:45:38,390 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 10:45:38,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:45:38,635 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 10:45:38,635 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:45:38,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 10:45:38,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 10:45:39,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-twwt.c.gcp-upstream.internal','N') ;"
2026-08-08 10:45:39,429 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:45:39,697 [I] Table x_portal_user does not exist in database ranger
2026-08-08 10:45:39,697 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:45:39,940 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 10:45:39,940 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:45:40,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:45:40,451 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 10:45:40,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 10:45:41.476 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.492 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.508 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.524 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.539 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.556 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.614 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.621 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.636 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.647 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.658 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.671 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.682 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.689 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.696 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:41.703 UTC [17425] WARNING: there is no transaction in progress
2026-08-08 10:45:42,376 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 10:45:42,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 10:45:42,648 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 10:45:42,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:45:42,913 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 10:45:42,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:45:43,174 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 10:45:43,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:45:43,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:45:43,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-twwt.c.gcp-upstream.internal';"
2026-08-08 10:45:43,931 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 10:45:43,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 10:45:44,187 [I] DB_PATCHES have already been applied
I20260808 10:45:44.193988 32694 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDI8wVd/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 10:45:44.194072 32694 mini_ranger.cc:193] Using host: 127.31.237.148
I20260808 10:45:44.199447 32694 mini_ranger.cc:227] Ranger admin URL: http://127.31.237.148:52779
Aug 08, 2026 10:45:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:45:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:45:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:45:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:45:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 10:45:45.611564 32694 mini_ranger.cc:160] Time spent starting Ranger: real 9.093s user 0.002s sys 0.011s
Aug 08, 2026 10:46:06 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:46:06 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 10:46:06 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:46:06 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:46:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:46:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:46:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:46:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:46:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:46:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:46:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:46:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:46:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:46:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:46:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:46:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:46:06 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:46:07 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 10:46:07.874405 32694 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.237.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.237.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/rangerkms_127.31.237.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/rangerkms_127.31.237.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.237.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.237.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.237.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/HTTP_127.31.237.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.237.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/HTTP_127.31.237.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 10:46:07 dist-test-slave-twwt krb5kdc[17094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185967, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 10:46:07.987255 32694 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 10:46:08.000584 32694 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 10:46:08.905299 32694 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 10:46:08.905413 32694 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-kms
I20260808 10:46:08.905537 32694 mini_ranger_kms.cc:227] Using postgres at 127.31.237.148:53891
2026-08-08 10:46:09,615 [I] DB FLAVOR :POSTGRES
2026-08-08 10:46:09,615 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:46:09,615 [I] Checking connection
2026-08-08 10:46:09,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 10:46:09,964 [I] connection success
2026-08-08 10:46:09,965 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 10:46:09,965 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 10:46:09,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 10:46:10,234 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 10:46:10,234 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:46:10,234 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 10:46:10,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:53891/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 10:46:10,511 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 10:46:11.012576 32694 mini_ranger_kms.cc:310] Created kms service
I20260808 10:46:11.233590 32694 mini_ranger_kms.cc:326] Created kudu user
I20260808 10:46:11.289086 32694 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 10:46:11 dist-test-slave-twwt krb5kdc[17094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185967, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:46:11.721660 32694 mini_ranger_kms.cc:384] Added ranger policy
I20260808 10:46:11.721834 32694 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 10:46:11.721858 32694 mini_ranger_kms.cc:240] Using host: 127.31.237.148
I20260808 10:46:11.727299 32694 mini_ranger_kms.cc:276] Ranger KMS PID: 17702
I20260808 10:46:11.727473 32694 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.237.148:58181
Aug 08, 2026 10:46:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:46:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:46:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:46:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:46:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:46:13 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 10:46:14 dist-test-slave-twwt krb5kdc[17094](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786185974, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.237.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:46:14.612972 32694 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.626s user 0.027s sys 0.625s
I20260808 10:46:14.613118 32694 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 10:46:14.613293 32694 mini_ranger_kms.cc:401] 127.31.237.148:58181/kms/v1/keys
Aug 08 10:46:14 dist-test-slave-twwt krb5kdc[17094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185974, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.237.148@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
Aug 08 10:46:15 dist-test-slave-twwt krb5kdc[17094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185974, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.237.148@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
Aug 08 10:46:17 dist-test-slave-twwt krb5kdc[17094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:46:17.855415 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:37139
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:37139
--ranger_config_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.237.148:58181/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:46:18.012487 17773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:46:18.012868 17773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:46:18.012915 17773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:46:18.017802 17773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:46:18.017874 17773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:46:18.017895 17773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:46:18.017911 17773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:46:18.017927 17773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:46:18.023926 17773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.237.148:58181/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:37139
--ranger_config_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:37139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:46:18.025115 17773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:46:18.026175 17773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:46:18.037231 17783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:46:18.037336 17786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:46:18.037307 17789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:46:18.037225 17773 server_base.cc:1034] running on GCE node
I20260808 10:46:18.038196 17773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:46:18.039384 17773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:46:18.040560 17773 hybrid_clock.cc:648] HybridClock initialized: now 1786185978040532 us; error 38 us; skew 500 ppm
Aug 08 10:46:18 dist-test-slave-twwt krb5kdc[17094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185978, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:46:18.048352 17773 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:46:18.049350 17773 webserver.cc:460] Webserver started at http://127.31.237.190:36329/ using document root <none> and password file <none>
I20260808 10:46:18.049957 17773 fs_manager.cc:359] Metadata directory not provided
I20260808 10:46:18.050016 17773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:46:18.050256 17773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:46:18 dist-test-slave-twwt krb5kdc[17094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185978, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:46:18.105204 17773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "0fcc6ac87fda4e4fad10cef96439b239"
format_stamp: "Formatted at 2026-08-08 10:46:18 on dist-test-slave-twwt"
server_key: "7f61b19322ff04a4a1f5368361fbd747"
server_key_iv: "e83c7d6c7078e5d2a77ed68c70e306de"
server_key_version: "kuduclusterkey@0"
I20260808 10:46:18.105999 17773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "0fcc6ac87fda4e4fad10cef96439b239"
format_stamp: "Formatted at 2026-08-08 10:46:18 on dist-test-slave-twwt"
server_key: "7f61b19322ff04a4a1f5368361fbd747"
server_key_iv: "e83c7d6c7078e5d2a77ed68c70e306de"
server_key_version: "kuduclusterkey@0"
I20260808 10:46:18.111390 17773 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 10:46:18.131060 17800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:46:18.132997 17773 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 10:46:18.133154 17773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "0fcc6ac87fda4e4fad10cef96439b239"
format_stamp: "Formatted at 2026-08-08 10:46:18 on dist-test-slave-twwt"
server_key: "7f61b19322ff04a4a1f5368361fbd747"
server_key_iv: "e83c7d6c7078e5d2a77ed68c70e306de"
server_key_version: "kuduclusterkey@0"
I20260808 10:46:18.133304 17773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:46:18.160815 17773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:46:18.163965 17773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:46:18.164297 17773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:46:18.176648 17773 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:37139
I20260808 10:46:18.176720 17878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:37139 every 8 connection(s)
I20260808 10:46:18.177804 17773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:46:18.181089 17880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:46:18.187572 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 17773
I20260808 10:46:18.187740 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:46:18.187975 17880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239: Bootstrap starting.
I20260808 10:46:18.188055 32694 external_mini_cluster.cc:1428] Setting key 554b9bb908d52e8e8bdf1ca94bd1fd6d
I20260808 10:46:18.191614 17880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239: Neither blocks nor log segments found. Creating new log.
I20260808 10:46:18.193259 17880 log.cc:824] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239: Log is configured to *not* fsync() on all Append() calls
I20260808 10:46:18.195765 17880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239: No bootstrap required, opened a new log
Aug 08 10:46:18 dist-test-slave-twwt krb5kdc[17094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:46:18.201036 17880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcc6ac87fda4e4fad10cef96439b239" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 37139 } }
I20260808 10:46:18.201331 17880 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:46:18.201416 17880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fcc6ac87fda4e4fad10cef96439b239, State: Initialized, Role: FOLLOWER
I20260808 10:46:18.201975 17880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcc6ac87fda4e4fad10cef96439b239" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 37139 } }
I20260808 10:46:18.202164 17880 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:46:18.202246 17880 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:46:18.202381 17880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:46:18.203847 17880 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcc6ac87fda4e4fad10cef96439b239" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 37139 } }
I20260808 10:46:18.204264 17880 leader_election.cc:302] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fcc6ac87fda4e4fad10cef96439b239; no voters:
I20260808 10:46:18.204753 17880 leader_election.cc:288] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:46:18.204913 17888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:46:18.205219 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:18.189611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49400 (local address 127.31.237.190:37139)
0808 10:46:18.190263 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0808 10:46:18.190294 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0808 10:46:18.190341 (+ 47us) server_negotiation.cc:407] Connection header received
0808 10:46:18.191251 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:46:18.191277 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:46:18.191751 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:46:18.192427 (+ 676us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:46:18.193005 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:18.194598 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:18.195478 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:18.196339 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:18.196579 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:18.199469 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:46:18.199476 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:46:18.199491 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:46:18.199522 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:46:18.201725 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:46:18.202345 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:46:18.202352 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:46:18.202359 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:46:18.202555 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:46:18.202938 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:46:18.202940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:46:18.202942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:46:18.203342 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:46:18.203570 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 10:46:18.203864 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 10:46:18.204166 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":326,"threads_started":1}
I20260808 10:46:18.205257 17888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [term 1 LEADER]: Becoming Leader. State: Replica: 0fcc6ac87fda4e4fad10cef96439b239, State: Running, Role: LEADER
I20260808 10:46:18.205859 17888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcc6ac87fda4e4fad10cef96439b239" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 37139 } }
I20260808 10:46:18.206214 17880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:46:18.208712 17890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fcc6ac87fda4e4fad10cef96439b239. Latest consensus state: current_term: 1 leader_uuid: "0fcc6ac87fda4e4fad10cef96439b239" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcc6ac87fda4e4fad10cef96439b239" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 37139 } } }
I20260808 10:46:18.208791 17889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fcc6ac87fda4e4fad10cef96439b239" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcc6ac87fda4e4fad10cef96439b239" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 37139 } } }
I20260808 10:46:18.208986 17890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [sys.catalog]: This master's current role is: LEADER
I20260808 10:46:18.209076 17889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239 [sys.catalog]: This master's current role is: LEADER
I20260808 10:46:18.211110 17906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:46:18.211328 17880 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:46:18.219740 17906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:46:18.228210 17906 catalog_manager.cc:1347] Generated new cluster ID: 3f5af1a734f4486198d39143658879ba
I20260808 10:46:18.228372 17906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:46:18.252120 17906 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:46:18.253930 17906 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:46:18.265980 17906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239: Generated new TSK 0
I20260808 10:46:18.267426 17906 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:46:19.103914 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.237.148:58181/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:37139
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:46:19.318302 17951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:46:19.318790 17951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:46:19.318920 17951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:46:19.325131 17951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:46:19.325227 17951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:46:19.325335 17951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:46:19.331287 17951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.237.148:58181/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:37139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:46:19.332448 17951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:46:19.333631 17951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:46:19.347546 17963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:46:19.347803 17951 server_base.cc:1034] running on GCE node
W20260808 10:46:19.347652 17962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:46:19.347572 17966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:46:19.348465 17951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:46:19.349205 17951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:46:19.350394 17951 hybrid_clock.cc:648] HybridClock initialized: now 1786185979350357 us; error 52 us; skew 500 ppm
Aug 08 10:46:19 dist-test-slave-twwt krb5kdc[17094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185979, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:46:19.353935 17951 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:46:19.355116 17951 webserver.cc:460] Webserver started at http://127.31.237.129:37203/ using document root <none> and password file <none>
I20260808 10:46:19.355731 17951 fs_manager.cc:359] Metadata directory not provided
I20260808 10:46:19.355823 17951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:46:19.356077 17951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:46:19 dist-test-slave-twwt krb5kdc[17094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185979, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:46:19.375710 17951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "478d1f455188419da325786e2da57b7f"
format_stamp: "Formatted at 2026-08-08 10:46:19 on dist-test-slave-twwt"
server_key: "eba5a43100cc05d79550290e4dbcb122"
server_key_iv: "d15313c9a2c35d3a5c91ba5b204ea1e4"
server_key_version: "kuduclusterkey@0"
I20260808 10:46:19.376571 17951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "478d1f455188419da325786e2da57b7f"
format_stamp: "Formatted at 2026-08-08 10:46:19 on dist-test-slave-twwt"
server_key: "eba5a43100cc05d79550290e4dbcb122"
server_key_iv: "d15313c9a2c35d3a5c91ba5b204ea1e4"
server_key_version: "kuduclusterkey@0"
I20260808 10:46:19.382323 17951 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 10:46:19.406008 17976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:46:19.408121 17951 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.001s
I20260808 10:46:19.408352 17951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "478d1f455188419da325786e2da57b7f"
format_stamp: "Formatted at 2026-08-08 10:46:19 on dist-test-slave-twwt"
server_key: "eba5a43100cc05d79550290e4dbcb122"
server_key_iv: "d15313c9a2c35d3a5c91ba5b204ea1e4"
server_key_version: "kuduclusterkey@0"
I20260808 10:46:19.408568 17951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:46:19.450139 17951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:46:19.454125 17951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:46:19.454349 17951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:46:19.455240 17951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:46:19.456398 17951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:46:19.456475 17951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:46:19.456590 17951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:46:19.456629 17951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:46:19.474785 17951 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:36181
I20260808 10:46:19.476119 17951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:46:19.478925 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 17951
I20260808 10:46:19.479071 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:46:19.479413 32694 external_mini_cluster.cc:1428] Setting key c18f8e1b2ae62ffdbf7a032467969b08
I20260808 10:46:19.494922 18141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:36181 every 8 connection(s)
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
Aug 08 10:46:19 dist-test-slave-twwt krb5kdc[17094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185979, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
I20260808 10:46:19.531483 18148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:19.505908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:37139 (local address 127.31.237.129:35509)
0808 10:46:19.506564 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0808 10:46:19.506657 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0808 10:46:19.507732 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:46:19.509630 (+ 1898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:46:19.509655 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:46:19.510075 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:46:19.511080 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:19.511105 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:19.512455 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:19.512459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:19.514211 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:19.514223 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:19.515186 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:19.515190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:19.515507 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:19.516412 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:46:19.516435 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:46:19.520790 (+ 4355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:46:19.527074 (+ 6284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:46:19.527091 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:46:19.527110 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:46:19.527555 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:46:19.527985 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:46:19.527989 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:46:19.527991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:46:19.528308 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:46:19.528945 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:46:19.528953 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:46:19.529352 (+ 399us) client_negotiation.cc:769] Sending connection context
0808 10:46:19.529608 (+ 256us) client_negotiation.cc:240] Negotiation successful
0808 10:46:19.530079 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":239,"threads_started":1}
I20260808 10:46:19.532413 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:19.507203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:35509 (local address 127.31.237.190:37139)
0808 10:46:19.509126 (+ 1923us) server_negotiation.cc:206] Beginning negotiation
0808 10:46:19.509131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:46:19.509149 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:46:19.509200 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:46:19.509203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:46:19.509255 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:46:19.511269 (+ 2014us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:46:19.511403 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:19.512286 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:19.514426 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:19.514854 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:19.514986 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:19.524464 (+ 9478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:46:19.524478 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:46:19.524481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:46:19.524515 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:46:19.526881 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:46:19.527737 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:46:19.527741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:46:19.527743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:46:19.527860 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:46:19.528461 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:46:19.528464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:46:19.528467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:46:19.528760 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:46:19.528866 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:46:19.532031 (+ 3165us) server_negotiation.cc:299] Negotiation successful
0808 10:46:19.532219 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1838,"thread_start_us":110,"threads_started":1}
I20260808 10:46:19.534440 18143 heartbeater.cc:347] Connected to a master server at 127.31.237.190:37139
I20260808 10:46:19.534821 18143 heartbeater.cc:464] Registering TS with master...
I20260808 10:46:19.535511 18143 heartbeater.cc:511] Master 127.31.237.190:37139 requested a full tablet report, sending...
I20260808 10:46:19.537645 17827 ts_manager.cc:194] Registered new tserver with Master: 478d1f455188419da325786e2da57b7f (127.31.237.129:36181)
I20260808 10:46:19.539463 17827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:35509
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:46:19.541821 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.237.148:58181/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:37139
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:46:19 dist-test-slave-twwt krb5kdc[17094](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786185979, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 10:46:19.753449 18151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:46:19.754122 18151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:46:19.754309 18151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:46:19.760428 18151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:46:19.760540 18151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:46:19.760648 18151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:46:19.767488 18151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.237.148:58181/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:37139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:46:19.768749 18151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:46:19.769914 18151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:46:19.783855 18158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:46:19.783955 18160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:46:19.784323 18151 server_base.cc:1034] running on GCE node
W20260808 10:46:19.784017 18163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:46:19.784984 18151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:46:19.785913 18151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:46:19.787196 18151 hybrid_clock.cc:648] HybridClock initialized: now 1786185979787147 us; error 55 us; skew 500 ppm
Aug 08 10:46:19 dist-test-slave-twwt krb5kdc[17094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185979, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:46:19.791838 18151 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:46:19.793252 18151 webserver.cc:460] Webserver started at http://127.31.237.130:35925/ using document root <none> and password file <none>
I20260808 10:46:19.794111 18151 fs_manager.cc:359] Metadata directory not provided
I20260808 10:46:19.794278 18151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:46:19.794601 18151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:46:19 dist-test-slave-twwt krb5kdc[17094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185979, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:46:19.817058 18151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "639d137d14694d9fa178576c6a508969"
format_stamp: "Formatted at 2026-08-08 10:46:19 on dist-test-slave-twwt"
server_key: "7c3ce8a3422f667bab4b14596127eff8"
server_key_iv: "f2dfc9081eba7540645eb438b21f37c7"
server_key_version: "kuduclusterkey@0"
I20260808 10:46:19.818030 18151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "639d137d14694d9fa178576c6a508969"
format_stamp: "Formatted at 2026-08-08 10:46:19 on dist-test-slave-twwt"
server_key: "7c3ce8a3422f667bab4b14596127eff8"
server_key_iv: "f2dfc9081eba7540645eb438b21f37c7"
server_key_version: "kuduclusterkey@0"
I20260808 10:46:19.827430 18151 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.001s
I20260808 10:46:19.846844 18173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:46:19.850456 18151 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260808 10:46:19.850747 18151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "639d137d14694d9fa178576c6a508969"
format_stamp: "Formatted at 2026-08-08 10:46:19 on dist-test-slave-twwt"
server_key: "7c3ce8a3422f667bab4b14596127eff8"
server_key_iv: "f2dfc9081eba7540645eb438b21f37c7"
server_key_version: "kuduclusterkey@0"
I20260808 10:46:19.851053 18151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:46:19.882025 18151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:46:19.886247 18151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:46:19.886570 18151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:46:19.887645 18151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:46:19.888917 18151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:46:19.889037 18151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:46:19.889159 18151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:46:19.889259 18151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:46:19.916600 18151 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:33403
I20260808 10:46:19.916687 18346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:33403 every 8 connection(s)
I20260808 10:46:19.918110 18151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:46:19.926923 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 18151
I20260808 10:46:19.927098 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:46:19.927482 32694 external_mini_cluster.cc:1428] Setting key 5616c28968054c5181613e734b0dc5d2
Aug 08 10:46:19 dist-test-slave-twwt krb5kdc[17094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185979, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:46:19.938936 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:19.922754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:53889 (local address 127.31.237.190:37139)
0808 10:46:19.922904 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:46:19.922909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:46:19.922930 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:46:19.923912 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:46:19.923917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:46:19.923977 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:46:19.924036 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:46:19.925956 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:19.926827 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:19.930595 (+ 3768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:19.930999 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:19.931085 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:19.934536 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:46:19.934540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:46:19.934542 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:46:19.934569 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:46:19.936382 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:46:19.937041 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:46:19.937045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:46:19.937047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:46:19.937138 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:46:19.937563 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:46:19.937566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:46:19.937568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:46:19.937805 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:46:19.937884 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:46:19.938589 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 10:46:19.938761 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:46:19.941123 18349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:19.921414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:37139 (local address 127.31.237.130:53889)
0808 10:46:19.922244 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0808 10:46:19.922306 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 10:46:19.923608 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:46:19.924469 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:46:19.924478 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:46:19.925016 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:46:19.925770 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:19.925792 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:19.928941 (+ 3149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:19.928944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:19.930445 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:19.930457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:19.931299 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:19.931303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:19.931587 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:19.932419 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:46:19.932442 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:46:19.934337 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:46:19.936530 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:46:19.936536 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:46:19.936553 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:46:19.936916 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:46:19.937256 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:46:19.937258 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:46:19.937260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:46:19.937446 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:46:19.937968 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:46:19.937974 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:46:19.938295 (+ 321us) client_negotiation.cc:769] Sending connection context
0808 10:46:19.939115 (+ 820us) client_negotiation.cc:240] Negotiation successful
0808 10:46:19.939565 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":296,"threads_started":1}
I20260808 10:46:19.942905 18347 heartbeater.cc:347] Connected to a master server at 127.31.237.190:37139
I20260808 10:46:19.943192 18347 heartbeater.cc:464] Registering TS with master...
I20260808 10:46:19.943817 18347 heartbeater.cc:511] Master 127.31.237.190:37139 requested a full tablet report, sending...
I20260808 10:46:19.945335 17827 ts_manager.cc:194] Registered new tserver with Master: 639d137d14694d9fa178576c6a508969 (127.31.237.130:33403)
I20260808 10:46:19.946118 17827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:53889
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:46:19.984519 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.237.148:58181/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:37139
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:46:20.200114 18352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:46:20.200570 18352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:46:20.200690 18352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:46:20.206524 18352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:46:20.206795 18352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:46:20.206943 18352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:46:20.214654 18352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.237.148:58181/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:37139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:46:20.216246 18352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:46:20.217685 18352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:46:20.247109 18363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:46:20.248384 18361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:46:20.248603 18367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:46:20.250041 18352 server_base.cc:1034] running on GCE node
I20260808 10:46:20.250716 18352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:46:20.251623 18352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Aug 08 10:46:20 dist-test-slave-twwt krb5kdc[17094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185979, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:46:20.267112 18352 hybrid_clock.cc:648] HybridClock initialized: now 1786185980266957 us; error 144 us; skew 500 ppm
Aug 08 10:46:20 dist-test-slave-twwt krb5kdc[17094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185980, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:46:20.271921 18352 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:46:20.273353 18352 webserver.cc:460] Webserver started at http://127.31.237.131:45333/ using document root <none> and password file <none>
I20260808 10:46:20.274163 18352 fs_manager.cc:359] Metadata directory not provided
I20260808 10:46:20.274317 18352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:46:20.274600 18352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:46:20 dist-test-slave-twwt krb5kdc[17094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185980, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:46:20.299984 18352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "9a768d818788459ea15a2ed5da11f1f3"
format_stamp: "Formatted at 2026-08-08 10:46:20 on dist-test-slave-twwt"
server_key: "de8638a76e5f57cb3db92056d757ab05"
server_key_iv: "497119f14c04e0fc44b6683e6913b92c"
server_key_version: "kuduclusterkey@0"
I20260808 10:46:20.300981 18352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "9a768d818788459ea15a2ed5da11f1f3"
format_stamp: "Formatted at 2026-08-08 10:46:20 on dist-test-slave-twwt"
server_key: "de8638a76e5f57cb3db92056d757ab05"
server_key_iv: "497119f14c04e0fc44b6683e6913b92c"
server_key_version: "kuduclusterkey@0"
I20260808 10:46:20.312995 18352 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.000s sys 0.004s
I20260808 10:46:20.331054 18379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:46:20.332906 18352 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260808 10:46:20.333133 18352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "9a768d818788459ea15a2ed5da11f1f3"
format_stamp: "Formatted at 2026-08-08 10:46:20 on dist-test-slave-twwt"
server_key: "de8638a76e5f57cb3db92056d757ab05"
server_key_iv: "497119f14c04e0fc44b6683e6913b92c"
server_key_version: "kuduclusterkey@0"
I20260808 10:46:20.333387 18352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:46:20.371793 18352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:46:20.376199 18352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:46:20.376458 18352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:46:20.377465 18352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:46:20.378917 18352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:46:20.379029 18352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:46:20.379119 18352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:46:20.379171 18352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:46:20.400914 18352 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:34901
I20260808 10:46:20.401080 18546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:34901 every 8 connection(s)
I20260808 10:46:20.402372 18352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:46:20.403177 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 18352
I20260808 10:46:20.403329 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:46:20.403680 32694 external_mini_cluster.cc:1428] Setting key f4ac128d44757de117930a7cfd7d812f
Aug 08 10:46:20 dist-test-slave-twwt krb5kdc[17094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185980, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:46:20.419807 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:20.406076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:38883 (local address 127.31.237.190:37139)
0808 10:46:20.406217 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:46:20.406223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:46:20.407533 (+ 1310us) server_negotiation.cc:407] Connection header received
0808 10:46:20.408942 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:46:20.408948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:46:20.409009 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:46:20.409080 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:46:20.410477 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.411153 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.412499 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.412745 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.412862 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:20.415935 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:46:20.415938 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:46:20.415940 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:46:20.415965 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:46:20.417576 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:46:20.418127 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:46:20.418131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:46:20.418133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:46:20.418219 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:46:20.418585 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:46:20.418590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:46:20.418592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:46:20.418833 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:46:20.418945 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:46:20.419486 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 10:46:20.419621 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:46:20.421208 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:20.406667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:37139 (local address 127.31.237.131:38883)
0808 10:46:20.407314 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0808 10:46:20.407436 (+ 122us) client_negotiation.cc:174] Beginning negotiation
0808 10:46:20.408653 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:46:20.409291 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:46:20.409298 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:46:20.409635 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:46:20.410325 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:20.410342 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.411313 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.411316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:20.412362 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:20.412372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.412911 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.412916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:20.413163 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:20.413898 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:46:20.413963 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:46:20.415757 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:46:20.417704 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:46:20.417711 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:46:20.417728 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:46:20.418016 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:46:20.418316 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:46:20.418321 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:46:20.418323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:46:20.418485 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:46:20.418946 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:46:20.418954 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:46:20.419228 (+ 274us) client_negotiation.cc:769] Sending connection context
0808 10:46:20.419442 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 10:46:20.419778 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":196,"threads_started":1}
I20260808 10:46:20.423583 18547 heartbeater.cc:347] Connected to a master server at 127.31.237.190:37139
I20260808 10:46:20.423892 18547 heartbeater.cc:464] Registering TS with master...
I20260808 10:46:20.424445 18547 heartbeater.cc:511] Master 127.31.237.190:37139 requested a full tablet report, sending...
I20260808 10:46:20.425585 17827 ts_manager.cc:194] Registered new tserver with Master: 9a768d818788459ea15a2ed5da11f1f3 (127.31.237.131:34901)
I20260808 10:46:20.426220 17827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:38883
I20260808 10:46:20.437041 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:46:20.448360 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:20.439861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49414 (local address 127.31.237.190:37139)
0808 10:46:20.440193 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0808 10:46:20.440197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:46:20.440450 (+ 253us) server_negotiation.cc:407] Connection header received
0808 10:46:20.440612 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:46:20.440615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:46:20.440670 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:46:20.440754 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:46:20.441626 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.442344 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.443262 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.443493 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.443601 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:20.445051 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:46:20.445055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:46:20.445058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:46:20.445083 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:46:20.446586 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:46:20.447046 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:46:20.447050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:46:20.447052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:46:20.447135 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:46:20.447429 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:46:20.447433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:46:20.447435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:46:20.447611 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:46:20.447751 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 10:46:20.448052 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 10:46:20.448162 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239}
I20260808 10:46:20.451483 17827 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:46:20.454960 17827 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:46:20.481583 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:20.467467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:34901 (local address 127.0.0.1:44612)
0808 10:46:20.468556 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0808 10:46:20.468606 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 10:46:20.468760 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:46:20.471183 (+ 2423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:46:20.471186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:46:20.471204 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:46:20.471398 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:20.471405 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.473170 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.473174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:20.474277 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:20.474286 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.480935 (+ 6649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.480939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:20.481111 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:20.481170 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 10:46:20.481265 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 10:46:20.481441 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"thread_start_us":157,"threads_started":1}
I20260808 10:46:20.482698 18568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:20.467655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:33403 (local address 127.0.0.1:33204)
0808 10:46:20.469076 (+ 1421us) negotiation.cc:107] Waiting for socket to connect
0808 10:46:20.469090 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:46:20.469202 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:46:20.470152 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:46:20.470158 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:46:20.470183 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:46:20.470326 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:20.470334 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.472664 (+ 2330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.472668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:20.474153 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:20.474164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.482309 (+ 8145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.482314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:20.482402 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:20.482424 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:46:20.482479 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 10:46:20.482551 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1360,"thread_start_us":99,"threads_started":1}
I20260808 10:46:20.482827 18571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:20.469052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44612 (local address 127.31.237.131:34901)
0808 10:46:20.470676 (+ 1624us) server_negotiation.cc:206] Beginning negotiation
0808 10:46:20.470698 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 10:46:20.470721 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:46:20.470796 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:46:20.470801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:46:20.470984 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:46:20.471549 (+ 565us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:46:20.471677 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.473038 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.474560 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.475507 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.482273 (+ 6766us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:20.482448 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 10:46:20.482506 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 10:46:20.482629 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1494,"thread_start_us":122,"threads_started":1}
I20260808 10:46:20.483615 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:20.467871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33204 (local address 127.31.237.130:33403)
0808 10:46:20.469461 (+ 1590us) server_negotiation.cc:206] Beginning negotiation
0808 10:46:20.469469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:46:20.469494 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:46:20.469576 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:46:20.469582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:46:20.469794 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:46:20.469959 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:46:20.470457 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.471689 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.481301 (+ 9612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.482169 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.483153 (+ 984us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:20.483287 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 10:46:20.483341 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 10:46:20.483434 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1470,"thread_start_us":76,"threads_started":1}
I20260808 10:46:20.486428 18451 tablet_service.cc:1467] Processing CreateTablet for tablet d9f56b3dbbed4e2cbfbc96372daa323d (DEFAULT_TABLE table=test-table [id=12828ed1dbe84ddf93ac9a9ee1911021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:46:20.487283 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:20.467467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:36181 (local address 127.0.0.1:55956)
0808 10:46:20.475773 (+ 8306us) negotiation.cc:107] Waiting for socket to connect
0808 10:46:20.475787 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:46:20.475924 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:46:20.476791 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:46:20.476795 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:46:20.476814 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:46:20.476897 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:20.476904 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.478338 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.478341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:20.479562 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:20.479573 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.486822 (+ 7249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.486827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:20.486933 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:20.486973 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 10:46:20.487059 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:46:20.487147 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8240,"mutex_wait_us":1857,"thread_start_us":110,"threads_started":1}
I20260808 10:46:20.488055 18451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9f56b3dbbed4e2cbfbc96372daa323d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:46:20.487844 18234 tablet_service.cc:1467] Processing CreateTablet for tablet d9f56b3dbbed4e2cbfbc96372daa323d (DEFAULT_TABLE table=test-table [id=12828ed1dbe84ddf93ac9a9ee1911021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:46:20.488731 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:20.468570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55956 (local address 127.31.237.129:36181)
0808 10:46:20.470258 (+ 1688us) server_negotiation.cc:206] Beginning negotiation
0808 10:46:20.470265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:46:20.476037 (+ 5772us) server_negotiation.cc:407] Connection header received
0808 10:46:20.476124 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:46:20.476128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:46:20.476302 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:46:20.476561 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:46:20.477031 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.478202 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.479703 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.480572 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.480688 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:20.480835 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:46:20.487553 (+ 6718us) server_negotiation.cc:299] Negotiation successful
0808 10:46:20.487690 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1570,"thread_start_us":182,"threads_started":1}
I20260808 10:46:20.488999 18234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9f56b3dbbed4e2cbfbc96372daa323d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:46:20.492022 18045 tablet_service.cc:1467] Processing CreateTablet for tablet d9f56b3dbbed4e2cbfbc96372daa323d (DEFAULT_TABLE table=test-table [id=12828ed1dbe84ddf93ac9a9ee1911021]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:46:20.493480 18045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9f56b3dbbed4e2cbfbc96372daa323d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:46:20.504653 18575 tablet_bootstrap.cc:492] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3: Bootstrap starting.
I20260808 10:46:20.506392 18577 tablet_bootstrap.cc:492] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f: Bootstrap starting.
I20260808 10:46:20.508792 18575 tablet_bootstrap.cc:654] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3: Neither blocks nor log segments found. Creating new log.
I20260808 10:46:20.509312 18577 tablet_bootstrap.cc:654] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f: Neither blocks nor log segments found. Creating new log.
I20260808 10:46:20.510380 18577 log.cc:824] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:46:20.510376 18575 log.cc:824] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3: Log is configured to *not* fsync() on all Append() calls
I20260808 10:46:20.511090 18579 tablet_bootstrap.cc:492] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969: Bootstrap starting.
I20260808 10:46:20.513279 18575 tablet_bootstrap.cc:492] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3: No bootstrap required, opened a new log
I20260808 10:46:20.513509 18575 ts_tablet_manager.cc:1397] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260808 10:46:20.516119 18579 tablet_bootstrap.cc:654] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969: Neither blocks nor log segments found. Creating new log.
I20260808 10:46:20.517503 18579 log.cc:824] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969: Log is configured to *not* fsync() on all Append() calls
I20260808 10:46:20.519233 18575 raft_consensus.cc:348] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478d1f455188419da325786e2da57b7f" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 36181 } } peers { permanent_uuid: "9a768d818788459ea15a2ed5da11f1f3" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34901 } } peers { permanent_uuid: "639d137d14694d9fa178576c6a508969" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33403 } }
I20260808 10:46:20.519671 18575 raft_consensus.cc:374] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:46:20.519807 18575 raft_consensus.cc:729] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a768d818788459ea15a2ed5da11f1f3, State: Initialized, Role: FOLLOWER
I20260808 10:46:20.520231 18577 tablet_bootstrap.cc:492] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f: No bootstrap required, opened a new log
I20260808 10:46:20.520465 18577 ts_tablet_manager.cc:1397] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20260808 10:46:20.520606 18575 consensus_queue.cc:260] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478d1f455188419da325786e2da57b7f" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 36181 } } peers { permanent_uuid: "9a768d818788459ea15a2ed5da11f1f3" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34901 } } peers { permanent_uuid: "639d137d14694d9fa178576c6a508969" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33403 } }
I20260808 10:46:20.521988 18575 ts_tablet_manager.cc:1428] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260808 10:46:20.522541 18547 heartbeater.cc:503] Master 127.31.237.190:37139 was elected leader, sending a full tablet report...
I20260808 10:46:20.524996 18579 tablet_bootstrap.cc:492] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969: No bootstrap required, opened a new log
I20260808 10:46:20.525290 18579 ts_tablet_manager.cc:1397] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260808 10:46:20.531991 18577 raft_consensus.cc:348] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478d1f455188419da325786e2da57b7f" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 36181 } } peers { permanent_uuid: "9a768d818788459ea15a2ed5da11f1f3" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34901 } } peers { permanent_uuid: "639d137d14694d9fa178576c6a508969" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33403 } }
I20260808 10:46:20.532483 18577 raft_consensus.cc:374] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:46:20.532596 18577 raft_consensus.cc:729] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478d1f455188419da325786e2da57b7f, State: Initialized, Role: FOLLOWER
I20260808 10:46:20.533263 18577 consensus_queue.cc:260] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478d1f455188419da325786e2da57b7f" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 36181 } } peers { permanent_uuid: "9a768d818788459ea15a2ed5da11f1f3" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34901 } } peers { permanent_uuid: "639d137d14694d9fa178576c6a508969" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33403 } }
I20260808 10:46:20.533241 18579 raft_consensus.cc:348] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478d1f455188419da325786e2da57b7f" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 36181 } } peers { permanent_uuid: "9a768d818788459ea15a2ed5da11f1f3" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34901 } } peers { permanent_uuid: "639d137d14694d9fa178576c6a508969" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33403 } }
I20260808 10:46:20.533643 18579 raft_consensus.cc:374] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:46:20.533773 18579 raft_consensus.cc:729] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639d137d14694d9fa178576c6a508969, State: Initialized, Role: FOLLOWER
I20260808 10:46:20.534355 18579 consensus_queue.cc:260] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478d1f455188419da325786e2da57b7f" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 36181 } } peers { permanent_uuid: "9a768d818788459ea15a2ed5da11f1f3" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34901 } } peers { permanent_uuid: "639d137d14694d9fa178576c6a508969" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33403 } }
I20260808 10:46:20.534770 18577 ts_tablet_manager.cc:1428] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f: Time spent starting tablet: real 0.014s user 0.010s sys 0.001s
I20260808 10:46:20.535239 18143 heartbeater.cc:503] Master 127.31.237.190:37139 was elected leader, sending a full tablet report...
I20260808 10:46:20.535735 18347 heartbeater.cc:503] Master 127.31.237.190:37139 was elected leader, sending a full tablet report...
I20260808 10:46:20.536500 18579 ts_tablet_manager.cc:1428] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
Aug 08 10:46:20 dist-test-slave-twwt krb5kdc[17094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185979, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
W20260808 10:46:20.655385 18549 tablet.cc:2369] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:46:20.683285 18348 tablet.cc:2369] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:46:20.761390 18144 tablet.cc:2369] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:46:20.812893 18587 raft_consensus.cc:482] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:46:20.813174 18587 raft_consensus.cc:504] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478d1f455188419da325786e2da57b7f" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 36181 } } peers { permanent_uuid: "9a768d818788459ea15a2ed5da11f1f3" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34901 } } peers { permanent_uuid: "639d137d14694d9fa178576c6a508969" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33403 } }
I20260808 10:46:20.815091 18587 leader_election.cc:288] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a768d818788459ea15a2ed5da11f1f3 (127.31.237.131:34901), 639d137d14694d9fa178576c6a508969 (127.31.237.130:33403)
I20260808 10:46:20.830549 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:20.816074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:54541 (local address 127.31.237.130:33403)
0808 10:46:20.816218 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:46:20.816225 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:46:20.821521 (+ 5296us) server_negotiation.cc:407] Connection header received
0808 10:46:20.821766 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:46:20.821771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:46:20.821827 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:46:20.821878 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:46:20.822473 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.823339 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.828959 (+ 5620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.829670 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.829777 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:20.829838 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:46:20.830303 (+ 465us) server_negotiation.cc:299] Negotiation successful
0808 10:46:20.830395 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:46:20.831902 18593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:20.815527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:33403 (local address 127.31.237.129:54541)
0808 10:46:20.821375 (+ 5848us) negotiation.cc:107] Waiting for socket to connect
0808 10:46:20.821394 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:46:20.821626 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:46:20.822031 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:46:20.822034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:46:20.822052 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:46:20.822320 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:20.822328 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.823549 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.823552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:20.825044 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:20.825057 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.829996 (+ 4939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.830000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:20.830123 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:20.830151 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 10:46:20.831235 (+ 1084us) client_negotiation.cc:240] Negotiation successful
0808 10:46:20.831340 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5745,"thread_start_us":150,"threads_started":1}
I20260808 10:46:20.833096 18274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9f56b3dbbed4e2cbfbc96372daa323d" candidate_uuid: "478d1f455188419da325786e2da57b7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639d137d14694d9fa178576c6a508969" is_pre_election: true
I20260808 10:46:20.833508 18274 raft_consensus.cc:2454] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478d1f455188419da325786e2da57b7f in term 0.
I20260808 10:46:20.834165 17979 leader_election.cc:302] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478d1f455188419da325786e2da57b7f, 639d137d14694d9fa178576c6a508969; no voters:
I20260808 10:46:20.834565 18587 raft_consensus.cc:2781] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:46:20.834687 18587 raft_consensus.cc:482] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:46:20.834743 18587 raft_consensus.cc:3037] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:46:20.836686 18587 raft_consensus.cc:504] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478d1f455188419da325786e2da57b7f" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 36181 } } peers { permanent_uuid: "9a768d818788459ea15a2ed5da11f1f3" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34901 } } peers { permanent_uuid: "639d137d14694d9fa178576c6a508969" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33403 } }
I20260808 10:46:20.837097 18587 leader_election.cc:288] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [CANDIDATE]: Term 1 election: Requested vote from peers 9a768d818788459ea15a2ed5da11f1f3 (127.31.237.131:34901), 639d137d14694d9fa178576c6a508969 (127.31.237.130:33403)
I20260808 10:46:20.838124 18274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9f56b3dbbed4e2cbfbc96372daa323d" candidate_uuid: "478d1f455188419da325786e2da57b7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639d137d14694d9fa178576c6a508969"
I20260808 10:46:20.838256 18274 raft_consensus.cc:3037] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:46:20.840466 18274 raft_consensus.cc:2454] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478d1f455188419da325786e2da57b7f in term 1.
I20260808 10:46:20.842115 17979 leader_election.cc:302] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478d1f455188419da325786e2da57b7f, 639d137d14694d9fa178576c6a508969; no voters:
I20260808 10:46:20.842372 18587 raft_consensus.cc:2781] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:46:20.842772 18587 raft_consensus.cc:686] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [term 1 LEADER]: Becoming Leader. State: Replica: 478d1f455188419da325786e2da57b7f, State: Running, Role: LEADER
I20260808 10:46:20.843183 18587 consensus_queue.cc:237] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478d1f455188419da325786e2da57b7f" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 36181 } } peers { permanent_uuid: "9a768d818788459ea15a2ed5da11f1f3" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34901 } } peers { permanent_uuid: "639d137d14694d9fa178576c6a508969" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33403 } }
I20260808 10:46:20.847165 17826 catalog_manager.cc:5495] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 478d1f455188419da325786e2da57b7f (127.31.237.129). New cstate: current_term: 1 leader_uuid: "478d1f455188419da325786e2da57b7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478d1f455188419da325786e2da57b7f" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 36181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a768d818788459ea15a2ed5da11f1f3" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639d137d14694d9fa178576c6a508969" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33403 } health_report { overall_health: UNKNOWN } } }
I20260808 10:46:20.848825 18571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:20.816651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:54141 (local address 127.31.237.131:34901)
0808 10:46:20.816799 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:46:20.816804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:46:20.825536 (+ 8732us) server_negotiation.cc:407] Connection header received
0808 10:46:20.825744 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:46:20.825747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:46:20.825801 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:46:20.825848 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:46:20.826304 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.827037 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.832632 (+ 5595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.833297 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.833396 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:20.833445 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:46:20.848558 (+ 15113us) server_negotiation.cc:299] Negotiation successful
0808 10:46:20.848682 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:46:20.850174 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:20.816455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:34901 (local address 127.31.237.129:54141)
0808 10:46:20.825425 (+ 8970us) negotiation.cc:107] Waiting for socket to connect
0808 10:46:20.825441 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:46:20.825619 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:46:20.825980 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:46:20.825983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:46:20.825998 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:46:20.826159 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:20.826165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.827352 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.827356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:20.828793 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:46:20.828806 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.833692 (+ 4886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.833697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:46:20.848353 (+ 14656us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:20.848393 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 10:46:20.849405 (+ 1012us) client_negotiation.cc:240] Negotiation successful
0808 10:46:20.849504 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8887,"thread_start_us":119,"threads_started":1}
I20260808 10:46:20.851135 18480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9f56b3dbbed4e2cbfbc96372daa323d" candidate_uuid: "478d1f455188419da325786e2da57b7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a768d818788459ea15a2ed5da11f1f3" is_pre_election: true
I20260808 10:46:20.851466 18480 raft_consensus.cc:2454] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478d1f455188419da325786e2da57b7f in term 0.
I20260808 10:46:20.851879 18475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9f56b3dbbed4e2cbfbc96372daa323d" candidate_uuid: "478d1f455188419da325786e2da57b7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a768d818788459ea15a2ed5da11f1f3"
I20260808 10:46:20.851997 18475 raft_consensus.cc:3037] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:46:20.853899 18475 raft_consensus.cc:2454] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478d1f455188419da325786e2da57b7f in term 1.
I20260808 10:46:20.944473 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:20.939270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55970 (local address 127.31.237.129:36181)
0808 10:46:20.939431 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 10:46:20.939436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:46:20.939451 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:46:20.939541 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:46:20.939544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:46:20.939608 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:46:20.939711 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:46:20.940130 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.940959 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.942190 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:46:20.942528 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:46:20.942649 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:46:20.943116 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:46:20.943699 (+ 583us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:46:20.943833 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 10:46:20.944239 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 10:46:20.944302 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:46:20.952435 18274 raft_consensus.cc:1261] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969 [term 1 FOLLOWER]: Refusing update from remote peer 478d1f455188419da325786e2da57b7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:46:20.952629 18475 raft_consensus.cc:1261] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3 [term 1 FOLLOWER]: Refusing update from remote peer 478d1f455188419da325786e2da57b7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:46:20.953282 18595 consensus_queue.cc:1035] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "639d137d14694d9fa178576c6a508969" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:46:20.953567 18587 consensus_queue.cc:1035] T d9f56b3dbbed4e2cbfbc96372daa323d P 478d1f455188419da325786e2da57b7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a768d818788459ea15a2ed5da11f1f3" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 34901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:46:20.967700 18613 mvcc.cc:205] Tried to move back new op lower bound from 7316217777972666368 to 7316217777537601536. Current Snapshot: MvccSnapshot[applied={T|T < 7316217777972666368}]
I20260808 10:46:20.969419 18611 mvcc.cc:205] Tried to move back new op lower bound from 7316217777972666368 to 7316217777537601536. Current Snapshot: MvccSnapshot[applied={T|T < 7316217777972666368}]
I20260808 10:46:20.980276 17826 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:46:20.980599 17826 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:46:20.983915 17826 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fcc6ac87fda4e4fad10cef96439b239: Sending DeleteTablet for 3 replicas of tablet d9f56b3dbbed4e2cbfbc96372daa323d
I20260808 10:46:20.986965 18451 tablet_service.cc:1514] Processing DeleteTablet for tablet d9f56b3dbbed4e2cbfbc96372daa323d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:46:20 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:44612
I20260808 10:46:20.987731 18234 tablet_service.cc:1514] Processing DeleteTablet for tablet d9f56b3dbbed4e2cbfbc96372daa323d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:46:20 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:33204
I20260808 10:46:20.988991 18045 tablet_service.cc:1514] Processing DeleteTablet for tablet d9f56b3dbbed4e2cbfbc96372daa323d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:46:20 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:55956
I20260808 10:46:20.990060 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 17951
I20260808 10:46:21.006341 18625 tablet_replica.cc:330] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969: stopping tablet replica
I20260808 10:46:21.006857 18625 raft_consensus.cc:2229] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:46:21.006935 18624 tablet_replica.cc:330] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3: stopping tablet replica
I20260808 10:46:21.007472 18625 raft_consensus.cc:2258] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:46:21.007297 18624 raft_consensus.cc:2229] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:46:21.008409 18624 raft_consensus.cc:2258] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:46:21.010004 18625 ts_tablet_manager.cc:1905] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:46:21.013214 18625 ts_tablet_manager.cc:1918] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:46:21.013428 18625 log.cc:1196] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/d9f56b3dbbed4e2cbfbc96372daa323d
I20260808 10:46:21.013868 18625 ts_tablet_manager.cc:1939] T d9f56b3dbbed4e2cbfbc96372daa323d P 639d137d14694d9fa178576c6a508969: Deleting consensus metadata
I20260808 10:46:21.015625 17803 catalog_manager.cc:4841] TS 639d137d14694d9fa178576c6a508969 (127.31.237.130:33403): tablet d9f56b3dbbed4e2cbfbc96372daa323d (table test-table [id=12828ed1dbe84ddf93ac9a9ee1911021]) successfully deleted
W20260808 10:46:21.016978 17801 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:36181 (error 108)
I20260808 10:46:21.017596 18624 ts_tablet_manager.cc:1905] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:46:21.018038 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 18151
I20260808 10:46:21.024313 18624 ts_tablet_manager.cc:1918] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:46:21.024441 18624 log.cc:1196] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/d9f56b3dbbed4e2cbfbc96372daa323d
I20260808 10:46:21.024763 18624 ts_tablet_manager.cc:1939] T d9f56b3dbbed4e2cbfbc96372daa323d P 9a768d818788459ea15a2ed5da11f1f3: Deleting consensus metadata
I20260808 10:46:21.026499 17804 catalog_manager.cc:4841] TS 9a768d818788459ea15a2ed5da11f1f3 (127.31.237.131:34901): tablet d9f56b3dbbed4e2cbfbc96372daa323d (table test-table [id=12828ed1dbe84ddf93ac9a9ee1911021]) successfully deleted
I20260808 10:46:21.027393 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:46:21.026756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:36181 (local address 127.0.0.1:55986)
0808 10:46:21.027184 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 10:46:21.027269 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:36181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":72,"threads_started":1}
W20260808 10:46:21.027650 17801 catalog_manager.cc:4584] TS 478d1f455188419da325786e2da57b7f (127.31.237.129:36181): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d9f56b3dbbed4e2cbfbc96372daa323d: Network error: Client connection negotiation failed: client connection to 127.31.237.129:36181: connect: Connection refused (error 111)
I20260808 10:46:21.032420 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 18352
I20260808 10:46:21.047485 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 17773
I20260808 10:46:21.067494 32694 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 10:46:21.424917 32694 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 10:46:21.425050 32694 mini_ranger.cc:66] Stopping Ranger...
I20260808 10:46:21.780295 32694 mini_ranger.cc:68] Stopped Ranger
2026-08-08 10:46:21.780 UTC [17139] LOG: received smart shutdown request
2026-08-08 10:46:21.784 UTC [17139] LOG: background worker "logical replication launcher" (PID 17150) exited with exit code 1
2026-08-08 10:46:21.785 UTC [17144] LOG: shutting down
2026-08-08 10:46:21.971 UTC [17139] LOG: database system is shut down
2026-08-08T10:46:21Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50854 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:46:22 dist-test-slave-twwt krb5kdc[18646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:46:22 dist-test-slave-twwt krb5kdc[18646](info): set up 2 sockets
Aug 08 10:46:22 dist-test-slave-twwt krb5kdc[18646](info): commencing operation
krb5kdc: starting...
W20260808 10:46:23.362324 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:46:23 dist-test-slave-twwt krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786185983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:46:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:46:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.237.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.237.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/rangeradmin_127.31.237.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/rangeradmin_127.31.237.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.237.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.237.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/rangerlookup_127.31.237.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/rangerlookup_127.31.237.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.237.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.237.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/HTTP_127.31.237.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/HTTP_127.31.237.148@KRBTEST.COM.keytab.
I20260808 10:46:23.557246 32694 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDI8wVd/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/postgres -l logfile start
2026-08-08 10:46:26.199 UTC [18687] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 10:46:26.199 UTC [18687] LOG: listening on IPv4 address "127.31.237.148", port 59999
2026-08-08 10:46:26.202 UTC [18687] LOG: listening on Unix socket "/tmp/.s.PGSQL.59999"
2026-08-08 10:46:26.226 UTC [18691] LOG: database system was shut down at 2026-08-08 10:46:24 UTC
2026-08-08 10:46:26.230 UTC [18687] LOG: database system is ready to accept connections
I20260808 10:46:27.403079 32694 mini_postgres.cc:96] Postgres bound to 59999
2026-08-08 10:46:27.411 UTC [18705] FATAL: database "slave" does not exist
127.31.237.148:59999 - accepting connections
I20260808 10:46:27.412220 32694 mini_ranger.cc:161] Starting Ranger...
I20260808 10:46:27.428472 32694 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 10:46:27.954964 32694 mini_ranger.cc:87] Created ranger Postgres database
I20260808 10:46:27.955067 32694 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-admin
2026-08-08 10:46:28,216 [I] DB FLAVOR :POSTGRES
2026-08-08 10:46:28,217 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:46:28,217 [I] Checking connection..
2026-08-08 10:46:28,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 10:46:28,469 [I] Checking connection passed.
2026-08-08 10:46:28,469 [I] --------- Verifying version history table ---------
2026-08-08 10:46:28,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:46:28,730 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:46:28,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:46:28,982 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:46:28,983 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 10:46:28,983 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 10:46:29.252 UTC [18800] WARNING: there is no transaction in progress
2026-08-08 10:46:29,260 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 10:46:29,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:46:29,513 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 10:46:29,513 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:46:29,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 10:46:29,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 10:46:30,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-twwt.c.gcp-upstream.internal','N') ;"
2026-08-08 10:46:30,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:46:30,551 [I] Table x_portal_user does not exist in database ranger
2026-08-08 10:46:30,551 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:46:30,835 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 10:46:30,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:46:31,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:46:31,359 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 10:46:31,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 10:46:32.411 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.429 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.445 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.461 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.480 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.498 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.557 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.565 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.582 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.593 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.604 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.618 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.628 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.635 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.641 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:32.649 UTC [18954] WARNING: there is no transaction in progress
2026-08-08 10:46:33,329 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 10:46:33,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 10:46:33,587 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 10:46:33,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:46:33,827 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 10:46:33,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:46:34,090 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 10:46:34,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:46:34,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:46:34,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-twwt.c.gcp-upstream.internal';"
2026-08-08 10:46:34,876 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 10:46:34,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 10:46:35,132 [I] DB_PATCHES have already been applied
I20260808 10:46:35.138561 32694 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDI8wVd/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 10:46:35.138697 32694 mini_ranger.cc:193] Using host: 127.31.237.148
I20260808 10:46:35.143652 32694 mini_ranger.cc:227] Ranger admin URL: http://127.31.237.148:55865
Aug 08, 2026 10:46:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:46:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:46:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:46:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:46:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:46:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:46:36 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 10:46:36.552677 32694 mini_ranger.cc:160] Time spent starting Ranger: real 9.140s user 0.007s sys 0.005s
Aug 08, 2026 10:46:57 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:46:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 10:46:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:46:57 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:46:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:46:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:46:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:46:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:46:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:46:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:46:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:46:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:46:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:46:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:46:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:46:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:46:57 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:46:58 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 10:46:59.391711 32694 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.237.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.237.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/rangerkms_127.31.237.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.237.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/rangerkms_127.31.237.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.237.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.237.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.237.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/HTTP_127.31.237.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.237.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/HTTP_127.31.237.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 10:46:59 dist-test-slave-twwt krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186019, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 10:46:59.505167 32694 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 10:46:59.517657 32694 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 10:47:00.435423 32694 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 10:47:00.435514 32694 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-kms
I20260808 10:47:00.435534 32694 mini_ranger_kms.cc:227] Using postgres at 127.31.237.148:59999
2026-08-08 10:47:00,615 [I] DB FLAVOR :POSTGRES
2026-08-08 10:47:00,615 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:47:00,615 [I] Checking connection
2026-08-08 10:47:00,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 10:47:00,873 [I] connection success
2026-08-08 10:47:00,873 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 10:47:00,873 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 10:47:00,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 10:47:01,124 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 10:47:01,124 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:47:01,125 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 10:47:01,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.237.148:59999/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 10:47:01,400 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 10:47:01.909080 32694 mini_ranger_kms.cc:310] Created kms service
I20260808 10:47:02.149453 32694 mini_ranger_kms.cc:326] Created kudu user
I20260808 10:47:02.197402 32694 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 10:47:02 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186019, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:47:02.627607 32694 mini_ranger_kms.cc:384] Added ranger policy
I20260808 10:47:02.627777 32694 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDI8wVd/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDI8wVd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 10:47:02.627801 32694 mini_ranger_kms.cc:240] Using host: 127.31.237.148
I20260808 10:47:02.632876 32694 mini_ranger_kms.cc:276] Ranger KMS PID: 19225
I20260808 10:47:02.632979 32694 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.237.148:56441
Aug 08, 2026 10:47:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:47:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:47:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:47:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:47:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:47:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 10:47:04 dist-test-slave-twwt krb5kdc[18646](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786186024, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.237.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 10:47:05 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186024, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.237.148@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:47:05.456954 32694 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.952s user 0.016s sys 0.150s
I20260808 10:47:05.457091 32694 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 10:47:05.457168 32694 mini_ranger_kms.cc:401] 127.31.237.148:56441/kms/v1/keys
Aug 08 10:47:05 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186024, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.237.148@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
Aug 08 10:47:08 dist-test-slave-twwt krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:47:08.088248 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:40123
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:40123
--ranger_config_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.237.148:56441/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:08.207295 19298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:08.207623 19298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:08.207703 19298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:08.211867 19298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:47:08.211966 19298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:08.212005 19298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:08.212062 19298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:47:08.212100 19298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:47:08.217962 19298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.237.148:56441/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:40123
--ranger_config_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:40123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:08.219255 19298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:08.220362 19298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:08.229607 19307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:08.229669 19305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:08.229668 19298 server_base.cc:1034] running on GCE node
W20260808 10:47:08.229621 19309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:08.230646 19298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:08.231799 19298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:08.232998 19298 hybrid_clock.cc:648] HybridClock initialized: now 1786186028232980 us; error 38 us; skew 500 ppm
Aug 08 10:47:08 dist-test-slave-twwt krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186028, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:08.241024 19298 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:47:08.242182 19298 webserver.cc:460] Webserver started at http://127.31.237.190:35025/ using document root <none> and password file <none>
I20260808 10:47:08.242859 19298 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:08.242923 19298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:08.243160 19298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:47:08 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186028, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:47:08.291364 19298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6"
format_stamp: "Formatted at 2026-08-08 10:47:08 on dist-test-slave-twwt"
server_key: "b44564e45eda4583b9753e5a78a892bb"
server_key_iv: "3896a21d78d1b4fe8b10f7e4d3594c74"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:08.292111 19298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6"
format_stamp: "Formatted at 2026-08-08 10:47:08 on dist-test-slave-twwt"
server_key: "b44564e45eda4583b9753e5a78a892bb"
server_key_iv: "3896a21d78d1b4fe8b10f7e4d3594c74"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:08.296341 19298 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:47:08.314415 19319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:08.315975 19298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:47:08.316120 19298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6"
format_stamp: "Formatted at 2026-08-08 10:47:08 on dist-test-slave-twwt"
server_key: "b44564e45eda4583b9753e5a78a892bb"
server_key_iv: "3896a21d78d1b4fe8b10f7e4d3594c74"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:08.316262 19298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:08.331774 19298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:08.335023 19298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:08.335244 19298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:08.344270 19298 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:40123
I20260808 10:47:08.344281 19391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:40123 every 8 connection(s)
I20260808 10:47:08.345338 19298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:47:08.347375 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 19298
I20260808 10:47:08.347517 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:47:08.347846 32694 external_mini_cluster.cc:1428] Setting key 9e6f4ece74f06fa9935f14705282b891
I20260808 10:47:08.348596 19393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:08.355974 19393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6: Bootstrap starting.
I20260808 10:47:08.358489 19393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6: Neither blocks nor log segments found. Creating new log.
Aug 08 10:47:08 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:08.359498 19393 log.cc:824] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:08.361531 19393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6: No bootstrap required, opened a new log
I20260808 10:47:08.365866 19393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40123 } }
I20260808 10:47:08.366117 19393 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:08.366204 19393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bee31ca13ad4cd2b29d3a06f2299fb6, State: Initialized, Role: FOLLOWER
I20260808 10:47:08.366312 19397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:08.349357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.31.237.190:40123)
0808 10:47:08.349855 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:08.349886 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:08.349928 (+ 42us) server_negotiation.cc:407] Connection header received
0808 10:47:08.350911 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:08.350936 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:08.351382 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:08.351788 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:08.352699 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:08.354205 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:08.355525 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:08.356215 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:08.356380 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:08.359589 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:08.359596 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:08.359614 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:08.359648 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:08.362791 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:08.363332 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:08.363339 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:08.363345 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:08.363488 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:08.363854 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:08.363858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:08.363859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:08.364407 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:08.364616 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:08.365028 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 10:47:08.365333 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":146,"threads_started":1}
I20260808 10:47:08.366837 19393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40123 } }
I20260808 10:47:08.367004 19393 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:47:08.367108 19393 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:47:08.367242 19393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:08.368690 19393 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40123 } }
I20260808 10:47:08.369105 19393 leader_election.cc:302] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bee31ca13ad4cd2b29d3a06f2299fb6; no voters:
I20260808 10:47:08.369536 19393 leader_election.cc:288] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:47:08.369663 19401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:47:08.369956 19401 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 1 LEADER]: Becoming Leader. State: Replica: 5bee31ca13ad4cd2b29d3a06f2299fb6, State: Running, Role: LEADER
I20260808 10:47:08.370371 19401 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40123 } }
I20260808 10:47:08.370992 19393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:47:08.372779 19403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bee31ca13ad4cd2b29d3a06f2299fb6. Latest consensus state: current_term: 1 leader_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40123 } } }
I20260808 10:47:08.372928 19403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:08.373376 19402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40123 } } }
I20260808 10:47:08.373479 19402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:08.373993 19393 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:47:08.375020 19418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:47:08.382781 19418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:47:08.392534 19418 catalog_manager.cc:1347] Generated new cluster ID: 95e11d3987c946738b3d49dfdad093d1
I20260808 10:47:08.392602 19418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:47:08.423362 19418 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:47:08.424947 19418 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:47:08.439564 19418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6: Generated new TSK 0
I20260808 10:47:08.440567 19418 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:47:09.154294 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.237.148:56441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40123
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:09.324079 19457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:09.324558 19457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:09.324654 19457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:09.330495 19457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:09.330555 19457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:09.330674 19457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:47:09.337181 19457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.237.148:56441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:09.338400 19457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:09.339605 19457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:09.351989 19466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:09.352471 19467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:09.353376 19469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:09.354180 19457 server_base.cc:1034] running on GCE node
I20260808 10:47:09.354694 19457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:09.355476 19457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:09.356668 19457 hybrid_clock.cc:648] HybridClock initialized: now 1786186029356634 us; error 48 us; skew 500 ppm
Aug 08 10:47:09 dist-test-slave-twwt krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186029, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:09.359997 19457 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:47:09.361012 19457 webserver.cc:460] Webserver started at http://127.31.237.129:46125/ using document root <none> and password file <none>
I20260808 10:47:09.361603 19457 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:09.361692 19457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:09.361948 19457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:47:09 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186029, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:47:09.383932 19457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "127fcec528074c9c8cb55eb7680e7221"
format_stamp: "Formatted at 2026-08-08 10:47:09 on dist-test-slave-twwt"
server_key: "4c429dba986e346ace5c593bac58d5ed"
server_key_iv: "3104aa1725b35a63c4b6a3737dfe29f4"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:09.384773 19457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "127fcec528074c9c8cb55eb7680e7221"
format_stamp: "Formatted at 2026-08-08 10:47:09 on dist-test-slave-twwt"
server_key: "4c429dba986e346ace5c593bac58d5ed"
server_key_iv: "3104aa1725b35a63c4b6a3737dfe29f4"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:09.390316 19457 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 10:47:09.419070 19481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:09.421123 19457 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.001s sys 0.000s
I20260808 10:47:09.421346 19457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "127fcec528074c9c8cb55eb7680e7221"
format_stamp: "Formatted at 2026-08-08 10:47:09 on dist-test-slave-twwt"
server_key: "4c429dba986e346ace5c593bac58d5ed"
server_key_iv: "3104aa1725b35a63c4b6a3737dfe29f4"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:09.421604 19457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:09.470067 19457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:09.474414 19457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:09.474725 19457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:09.475569 19457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:09.476778 19457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:09.476863 19457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:09.476946 19457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:09.476998 19457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:09.488131 19457 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:34163
I20260808 10:47:09.488154 19658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:34163 every 8 connection(s)
I20260808 10:47:09.489221 19457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:47:09.489888 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 19457
I20260808 10:47:09.490037 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:47:09.490375 32694 external_mini_cluster.cc:1428] Setting key 6668b790b2441e40e47673118672ffc7
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
Aug 08 10:47:09 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186029, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:09.512988 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:09.495801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:57085 (local address 127.31.237.190:40123)
0808 10:47:09.496533 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:09.496540 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:09.498503 (+ 1963us) server_negotiation.cc:407] Connection header received
0808 10:47:09.498561 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:09.498567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:09.498661 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:09.498834 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:09.500565 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:09.501468 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:09.503306 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:09.503712 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:09.503796 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:09.507892 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:09.507899 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:09.507906 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:09.507941 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:09.510059 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:09.510781 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:09.510788 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:09.510793 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:09.510893 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:09.511339 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:09.511346 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:09.511350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:09.511620 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:09.511715 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:09.512513 (+ 798us) server_negotiation.cc:299] Negotiation successful
0808 10:47:09.512681 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":92,"threads_started":1}
I20260808 10:47:09.515012 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:09.495336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:40123 (local address 127.31.237.129:57085)
0808 10:47:09.496829 (+ 1493us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:09.496889 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:09.498177 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:09.499178 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:09.499190 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:09.499584 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:09.500366 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:09.500392 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:09.501644 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:09.501651 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:09.503148 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:09.503161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:09.503972 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:09.503974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:09.504119 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:09.504897 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:09.504915 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:09.507670 (+ 2755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:09.510201 (+ 2531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:09.510211 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:09.510231 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:09.510639 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:09.511015 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:09.511021 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:09.511026 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:09.511219 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:09.511804 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:09.511813 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:09.512174 (+ 361us) client_negotiation.cc:769] Sending connection context
0808 10:47:09.513303 (+ 1129us) client_negotiation.cc:240] Negotiation successful
0808 10:47:09.513706 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"thread_start_us":145,"threads_started":1}
I20260808 10:47:09.516628 19659 heartbeater.cc:347] Connected to a master server at 127.31.237.190:40123
I20260808 10:47:09.516909 19659 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:09.517576 19659 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
I20260808 10:47:09.519351 19340 ts_manager.cc:194] Registered new tserver with Master: 127fcec528074c9c8cb55eb7680e7221 (127.31.237.129:34163)
I20260808 10:47:09.521054 19340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:57085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:47:09.530194 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.237.148:56441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40123
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:47:09 dist-test-slave-twwt krb5kdc[18646](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786186029, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 10:47:09.671676 19669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:09.672094 19669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:09.672206 19669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:09.676769 19669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:09.676859 19669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:09.676966 19669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:47:09.683833 19669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.237.148:56441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:09.685026 19669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:09.686097 19669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:09.700003 19679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:09.700003 19682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:09.700376 19678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:09.701341 19669 server_base.cc:1034] running on GCE node
I20260808 10:47:09.701853 19669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:09.702702 19669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:09.703872 19669 hybrid_clock.cc:648] HybridClock initialized: now 1786186029703793 us; error 97 us; skew 500 ppm
Aug 08 10:47:09 dist-test-slave-twwt krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186029, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:09.707435 19669 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:47:09.708580 19669 webserver.cc:460] Webserver started at http://127.31.237.130:46677/ using document root <none> and password file <none>
I20260808 10:47:09.709175 19669 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:09.709343 19669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:09.709666 19669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:47:09 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186029, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:47:09.735603 19669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "655a45d91fe548e1b15d506e74356b12"
format_stamp: "Formatted at 2026-08-08 10:47:09 on dist-test-slave-twwt"
server_key: "26f40e0106d66348a937e6a70c689f8d"
server_key_iv: "59373d693d90253838e3bb87de7ec441"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:09.736407 19669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "655a45d91fe548e1b15d506e74356b12"
format_stamp: "Formatted at 2026-08-08 10:47:09 on dist-test-slave-twwt"
server_key: "26f40e0106d66348a937e6a70c689f8d"
server_key_iv: "59373d693d90253838e3bb87de7ec441"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:09.742105 19669 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 10:47:09.775125 19696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:09.777340 19669 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.003s sys 0.000s
I20260808 10:47:09.777612 19669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "655a45d91fe548e1b15d506e74356b12"
format_stamp: "Formatted at 2026-08-08 10:47:09 on dist-test-slave-twwt"
server_key: "26f40e0106d66348a937e6a70c689f8d"
server_key_iv: "59373d693d90253838e3bb87de7ec441"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:09.777897 19669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:09.818456 19669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:09.822651 19669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:09.822954 19669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:09.823789 19669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:09.824945 19669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:09.825059 19669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:09.825172 19669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:09.825238 19669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:09.836504 19669 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:34345
I20260808 10:47:09.838133 19669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:47:09.840579 19875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:34345 every 8 connection(s)
I20260808 10:47:09.841995 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 19669
I20260808 10:47:09.842121 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:47:09.842438 32694 external_mini_cluster.cc:1428] Setting key 0cde242b2cfc4962831dcc8d2642b5a7
Aug 08 10:47:09 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186029, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:09.857441 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:09.843379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:53451 (local address 127.31.237.190:40123)
0808 10:47:09.843502 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:09.843506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:09.843596 (+ 90us) server_negotiation.cc:407] Connection header received
0808 10:47:09.844850 (+ 1254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:09.844854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:09.844910 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:09.844964 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:09.846698 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:09.847514 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:09.849247 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:09.849602 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:09.849713 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:09.852991 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:09.852995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:09.852998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:09.853026 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:09.854840 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:09.855470 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:09.855474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:09.855476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:09.855565 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:09.856001 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:09.856005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:09.856007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:09.856267 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:09.856350 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:09.857109 (+ 759us) server_negotiation.cc:299] Negotiation successful
0808 10:47:09.857261 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:47:09.859412 19879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:09.841852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:40123 (local address 127.31.237.130:53451)
0808 10:47:09.843433 (+ 1581us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:09.843492 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:09.844586 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:09.845416 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:09.845426 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:09.845806 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:09.846493 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:09.846513 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:09.847682 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:09.847685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:09.849099 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:09.849111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:09.849904 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:09.849908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:09.850123 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:09.850844 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:09.850867 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:09.852784 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:09.854988 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:09.854994 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:09.855012 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:09.855353 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:09.855677 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:09.855680 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:09.855683 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:09.855878 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:09.856450 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:09.856457 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:09.856805 (+ 348us) client_negotiation.cc:769] Sending connection context
0808 10:47:09.857885 (+ 1080us) client_negotiation.cc:240] Negotiation successful
0808 10:47:09.858264 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1302,"thread_start_us":177,"threads_started":1}
I20260808 10:47:09.860971 19877 heartbeater.cc:347] Connected to a master server at 127.31.237.190:40123
I20260808 10:47:09.861224 19877 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:09.861814 19877 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
I20260808 10:47:09.863095 19340 ts_manager.cc:194] Registered new tserver with Master: 655a45d91fe548e1b15d506e74356b12 (127.31.237.130:34345)
I20260808 10:47:09.863864 19340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:53451
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:47:09.891426 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.237.148:56441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40123
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:47:10 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186029, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
W20260808 10:47:10.043592 19883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:10.044063 19883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:10.044152 19883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:10.050379 19883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:10.050554 19883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:10.050727 19883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:47:10.058723 19883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.237.148:56441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:40123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:10.060288 19883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:10.061735 19883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:10.075624 19896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:10.075708 19893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:10.076884 19891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:10.077287 19883 server_base.cc:1034] running on GCE node
I20260808 10:47:10.077818 19883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:10.078677 19883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:10.086162 19883 hybrid_clock.cc:648] HybridClock initialized: now 1786186030084733 us; error 1422 us; skew 500 ppm
Aug 08 10:47:10 dist-test-slave-twwt krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186030, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:10.090775 19883 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:47:10.092108 19883 webserver.cc:460] Webserver started at http://127.31.237.131:34957/ using document root <none> and password file <none>
I20260808 10:47:10.092973 19883 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:10.093130 19883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:10.093444 19883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:47:10 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186030, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:47:10.113829 19883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "ae80d3b741fe42de9ac440198b85d556"
format_stamp: "Formatted at 2026-08-08 10:47:10 on dist-test-slave-twwt"
server_key: "0a2282a437c98ee4f9a3eabe86e5aa0d"
server_key_iv: "7c3e56610d2e2a99c1956e97c882af28"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:10.115237 19883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "ae80d3b741fe42de9ac440198b85d556"
format_stamp: "Formatted at 2026-08-08 10:47:10 on dist-test-slave-twwt"
server_key: "0a2282a437c98ee4f9a3eabe86e5aa0d"
server_key_iv: "7c3e56610d2e2a99c1956e97c882af28"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:10.122115 19883 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260808 10:47:10.144529 19909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:10.146477 19883 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.004s sys 0.000s
I20260808 10:47:10.146651 19883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "ae80d3b741fe42de9ac440198b85d556"
format_stamp: "Formatted at 2026-08-08 10:47:10 on dist-test-slave-twwt"
server_key: "0a2282a437c98ee4f9a3eabe86e5aa0d"
server_key_iv: "7c3e56610d2e2a99c1956e97c882af28"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:10.146797 19883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:10.173409 19883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:10.178092 19883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:10.178429 19883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:10.179414 19883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:10.180900 19883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:10.181033 19883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:10.181160 19883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:10.181241 19883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:10.207137 19883 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:36053
I20260808 10:47:10.209686 19883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:47:10.214768 20085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:36053 every 8 connection(s)
I20260808 10:47:10.218876 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 19883
I20260808 10:47:10.219007 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:47:10.219297 32694 external_mini_cluster.cc:1428] Setting key 2008a88e1de3a4ced389c094accf8027
Aug 08 10:47:10 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186030, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:10.235986 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:10.214957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:49141 (local address 127.31.237.190:40123)
0808 10:47:10.215075 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:10.215080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:10.215097 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:47:10.216749 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:10.216753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:10.216824 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:10.216932 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:10.223567 (+ 6635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:10.224411 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:10.226343 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:10.226712 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:10.226841 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:10.231445 (+ 4604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:10.231449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:10.231453 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:10.231484 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:10.233328 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:10.234018 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:10.234022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:10.234024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:10.234110 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:10.234523 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:10.234526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:10.234528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:10.234822 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:10.234913 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:10.235663 (+ 750us) server_negotiation.cc:299] Negotiation successful
0808 10:47:10.235817 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:47:10.237515 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:10.213567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:40123 (local address 127.31.237.131:49141)
0808 10:47:10.214472 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:10.214552 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:10.216481 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:10.217155 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:10.217166 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:10.217888 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:10.223378 (+ 5490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:10.223403 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:10.224591 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:10.224595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:10.226174 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:10.226188 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:10.226964 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:10.226968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:10.227311 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:10.228157 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:10.228179 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:10.230475 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:10.233471 (+ 2996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:10.233479 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:10.233498 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:10.233897 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:10.234227 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:10.234230 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:10.234233 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:10.234417 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:10.234984 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:10.234991 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:10.235341 (+ 350us) client_negotiation.cc:769] Sending connection context
0808 10:47:10.235560 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 10:47:10.236302 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":236,"threads_started":1}
I20260808 10:47:10.239019 20086 heartbeater.cc:347] Connected to a master server at 127.31.237.190:40123
I20260808 10:47:10.239305 20086 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:10.239943 20086 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
I20260808 10:47:10.241217 19340 ts_manager.cc:194] Registered new tserver with Master: ae80d3b741fe42de9ac440198b85d556 (127.31.237.131:36053)
I20260808 10:47:10.241990 19340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:49141
I20260808 10:47:10.247165 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:47:10.247941 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 19457
I20260808 10:47:10.294426 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 19669
I20260808 10:47:10.309616 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 19883
I20260808 10:47:10.322358 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 19298
I20260808 10:47:10.342571 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:40123
--webserver_interface=127.31.237.190
--webserver_port=35025
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:40123
--ranger_config_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.237.148:56441/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.237.148:56441/kms with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:10.469563 20090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:10.469857 20090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:10.469902 20090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:10.475209 20090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:47:10.475284 20090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:10.475307 20090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:10.475325 20090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:47:10.475389 20090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:47:10.481549 20090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.237.148:56441/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:40123
--ranger_config_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:40123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=35025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:10.482776 20090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:10.483901 20090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:10.493500 20105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:10.493427 20100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:10.493659 20090 server_base.cc:1034] running on GCE node
W20260808 10:47:10.493438 20101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:10.494378 20090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:10.495550 20090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:10.496724 20090 hybrid_clock.cc:648] HybridClock initialized: now 1786186030496687 us; error 51 us; skew 500 ppm
Aug 08 10:47:10 dist-test-slave-twwt krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186030, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:10.499804 20090 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:47:10.500751 20090 webserver.cc:460] Webserver started at http://127.31.237.190:35025/ using document root <none> and password file <none>
I20260808 10:47:10.501303 20090 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:10.501407 20090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:47:10 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186030, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:47:10.586534 20090 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 10:47:10.589466 20116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:10.591022 20090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:47:10.591223 20090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6"
format_stamp: "Formatted at 2026-08-08 10:47:08 on dist-test-slave-twwt"
server_key: "b44564e45eda4583b9753e5a78a892bb"
server_key_iv: "3896a21d78d1b4fe8b10f7e4d3594c74"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:10.591809 20090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:10.616761 20090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:10.620119 20090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:10.620338 20090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:10.639935 20090 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:40123
I20260808 10:47:10.639966 20186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:40123 every 8 connection(s)
I20260808 10:47:10.641229 20090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:47:10.642099 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 20090
I20260808 10:47:10.642673 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.237.148:56441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:34163
--local_ip_for_outbound_sockets=127.31.237.129
--tserver_master_addrs=127.31.237.190:40123
--webserver_port=46125
--webserver_interface=127.31.237.129
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.237.148:56441/kms with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:47:10.647030 20187 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:47:10.648896 20187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6: Bootstrap starting.
I20260808 10:47:10.672605 20187 log.cc:824] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:10.679507 20187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:47:10.679946 20187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6: Bootstrap complete.
I20260808 10:47:10.684263 20187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40123 } }
I20260808 10:47:10.684661 20187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bee31ca13ad4cd2b29d3a06f2299fb6, State: Initialized, Role: FOLLOWER
I20260808 10:47:10.685218 20187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40123 } }
I20260808 10:47:10.685338 20187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:47:10.685433 20187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:47:10.685550 20187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:47:10.687611 20187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40123 } }
I20260808 10:47:10.687951 20187 leader_election.cc:302] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bee31ca13ad4cd2b29d3a06f2299fb6; no voters:
I20260808 10:47:10.688299 20187 leader_election.cc:288] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:47:10.688477 20193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:47:10.688859 20193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [term 2 LEADER]: Becoming Leader. State: Replica: 5bee31ca13ad4cd2b29d3a06f2299fb6, State: Running, Role: LEADER
I20260808 10:47:10.689239 20193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40123 } }
I20260808 10:47:10.689616 20187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:47:10.690929 20196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bee31ca13ad4cd2b29d3a06f2299fb6. Latest consensus state: current_term: 2 leader_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40123 } } }
I20260808 10:47:10.691052 20196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:10.691401 20194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bee31ca13ad4cd2b29d3a06f2299fb6" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 40123 } } }
I20260808 10:47:10.691509 20194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:10.692101 20187 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:47:10.692193 20209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:47:10.699137 20209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:47:10.700102 20209 catalog_manager.cc:1259] Loaded cluster ID: 95e11d3987c946738b3d49dfdad093d1
I20260808 10:47:10.700174 20209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:47:10.703783 20209 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:47:10.704902 20209 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6: Loaded TSK: 0
I20260808 10:47:10.706388 20209 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:47:10.770419 20191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:10.770792 20191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:10.770951 20191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:10.775239 20191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:10.775331 20191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:10.775461 20191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:47:10.783488 20191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.237.148:56441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:34163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=46125
--tserver_master_addrs=127.31.237.190:40123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:10.785040 20191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:10.786588 20191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:10.800670 20243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:10.800879 20191 server_base.cc:1034] running on GCE node
W20260808 10:47:10.801159 20247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:10.801005 20244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:10.801753 20191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:10.802532 20191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:10.803732 20191 hybrid_clock.cc:648] HybridClock initialized: now 1786186030803709 us; error 39 us; skew 500 ppm
Aug 08 10:47:10 dist-test-slave-twwt krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186030, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:10.807528 20191 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:47:10.808736 20191 webserver.cc:460] Webserver started at http://127.31.237.129:46125/ using document root <none> and password file <none>
I20260808 10:47:10.809494 20191 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:10.809620 20191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:47:10 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186030, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:47:10.836648 20191 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.000s
I20260808 10:47:10.840189 20256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:10.841964 20191 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:47:10.842103 20191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "127fcec528074c9c8cb55eb7680e7221"
format_stamp: "Formatted at 2026-08-08 10:47:09 on dist-test-slave-twwt"
server_key: "4c429dba986e346ace5c593bac58d5ed"
server_key_iv: "3104aa1725b35a63c4b6a3737dfe29f4"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:10.843000 20191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:10.880728 20191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:10.884630 20191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:10.884856 20191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:10.885638 20191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:10.886734 20191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:10.886813 20191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:10.886938 20191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:10.886991 20191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:10.900421 20191 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:34163
I20260808 10:47:10.902596 20191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:47:10.904698 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 20191
I20260808 10:47:10.904942 20425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:34163 every 8 connection(s)
I20260808 10:47:10.905272 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.237.148:56441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:34345
--local_ip_for_outbound_sockets=127.31.237.130
--tserver_master_addrs=127.31.237.190:40123
--webserver_port=46677
--webserver_interface=127.31.237.130
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.237.148:56441/kms with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:47:10 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186030, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:10.932036 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:10.911280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:45163 (local address 127.31.237.190:40123)
0808 10:47:10.912007 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:10.912034 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:10.912575 (+ 541us) server_negotiation.cc:407] Connection header received
0808 10:47:10.914444 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:10.914464 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:10.914964 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:10.915351 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:10.917296 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:10.919038 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:10.920972 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:10.921674 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:10.921842 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:10.924834 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:10.924841 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:10.924859 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:10.924893 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:10.927024 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:10.927661 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:10.927667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:10.927674 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:10.927826 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:10.928204 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:10.928207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:10.928209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:10.928694 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:10.928829 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:10.930809 (+ 1980us) server_negotiation.cc:299] Negotiation successful
0808 10:47:10.931101 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":105,"threads_started":1}
I20260808 10:47:10.933101 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:10.910735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:40123 (local address 127.31.237.129:45163)
0808 10:47:10.912359 (+ 1624us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:10.912422 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:10.913772 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:10.915923 (+ 2151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:10.915934 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:10.916324 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:10.917064 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:10.917089 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:10.919221 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:10.919226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:10.920829 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:10.920841 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:10.922060 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:10.922064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:10.922264 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:10.922978 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:10.923000 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:10.924596 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:10.927162 (+ 2566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:10.927168 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:10.927184 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:10.927520 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:10.927933 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:10.927936 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:10.927938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:10.928102 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:10.928902 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:10.928912 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:10.929260 (+ 348us) client_negotiation.cc:769] Sending connection context
0808 10:47:10.929487 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 10:47:10.929837 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1366,"thread_start_us":140,"threads_started":1}
I20260808 10:47:10.936183 20426 heartbeater.cc:347] Connected to a master server at 127.31.237.190:40123
I20260808 10:47:10.936422 20426 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:10.937011 20426 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
W20260808 10:47:10.939350 20426 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:40123 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:47:10.939469 20426 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:10.939636 20426 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
I20260808 10:47:10.940523 20426 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:10.940687 20426 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
W20260808 10:47:10.941466 20426 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:47:11.059170 20428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:11.059460 20428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:11.059571 20428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:11.063808 20428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:11.063900 20428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:11.064006 20428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:47:11.069702 20428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.237.148:56441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:34345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=46677
--tserver_master_addrs=127.31.237.190:40123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:11.071034 20428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:11.072132 20428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:11.082971 20441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:11.083102 20445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:11.083179 20443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:11.083532 20428 server_base.cc:1034] running on GCE node
I20260808 10:47:11.084033 20428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:11.084831 20428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:11.085994 20428 hybrid_clock.cc:648] HybridClock initialized: now 1786186031085969 us; error 39 us; skew 500 ppm
Aug 08 10:47:11 dist-test-slave-twwt krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186031, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:11.089079 20428 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:47:11.090191 20428 webserver.cc:460] Webserver started at http://127.31.237.130:46677/ using document root <none> and password file <none>
I20260808 10:47:11.090911 20428 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:11.091110 20428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:47:11 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186031, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:47:11.115262 20428 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:47:11.118705 20455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:11.120505 20428 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 10:47:11.120646 20428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "655a45d91fe548e1b15d506e74356b12"
format_stamp: "Formatted at 2026-08-08 10:47:09 on dist-test-slave-twwt"
server_key: "26f40e0106d66348a937e6a70c689f8d"
server_key_iv: "59373d693d90253838e3bb87de7ec441"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:11.121443 20428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:11.146478 20428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:11.150647 20428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:11.150875 20428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:11.151732 20428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:11.152905 20428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:11.153029 20428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:11.153139 20428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:11.153209 20428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:11.165604 20428 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:34345
I20260808 10:47:11.166208 20620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:34345 every 8 connection(s)
I20260808 10:47:11.167188 20428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:47:11.176450 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 20428
I20260808 10:47:11.176932 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.237.148:56441/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:36053
--local_ip_for_outbound_sockets=127.31.237.131
--tserver_master_addrs=127.31.237.190:40123
--webserver_port=34957
--webserver_interface=127.31.237.131
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.237.148:56441/kms with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:47:11 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186031, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:11.185478 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:11.169373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:36037 (local address 127.31.237.190:40123)
0808 10:47:11.169525 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:11.169530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:11.171690 (+ 2160us) server_negotiation.cc:407] Connection header received
0808 10:47:11.172983 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:11.172987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:11.173057 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:11.173164 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:11.174920 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:11.175776 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:11.177564 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:11.177918 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:11.178016 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:11.181242 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:11.181246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:11.181248 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:11.181279 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:11.183092 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:11.183697 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:11.183700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:11.183701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:11.183788 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:11.184184 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:11.184189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:11.184193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:11.184411 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:11.184531 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:11.185122 (+ 591us) server_negotiation.cc:299] Negotiation successful
0808 10:47:11.185291 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:47:11.186441 20623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:11.169876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:40123 (local address 127.31.237.130:36037)
0808 10:47:11.171473 (+ 1597us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:11.171537 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:11.172676 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:11.173489 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:11.173500 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:11.173881 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:11.174709 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:11.174735 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:11.175961 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:11.175964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:11.177401 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:11.177413 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:11.178283 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:11.178292 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:11.178512 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:11.179320 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:11.179343 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:11.181003 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:11.183233 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:11.183240 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:11.183261 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:11.183566 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:11.183890 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:11.183893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:11.183895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:11.184074 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:11.184532 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:11.184541 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:11.184847 (+ 306us) client_negotiation.cc:769] Sending connection context
0808 10:47:11.185063 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 10:47:11.185431 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":149,"threads_started":1}
I20260808 10:47:11.187887 20621 heartbeater.cc:347] Connected to a master server at 127.31.237.190:40123
I20260808 10:47:11.188161 20621 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:11.188832 20621 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
W20260808 10:47:11.191056 20621 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:40123 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:47:11.191200 20621 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:11.191434 20621 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
I20260808 10:47:11.192526 20621 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:11.192734 20621 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
W20260808 10:47:11.193568 20621 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:47:11.325745 20626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:11.326064 20626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:11.326175 20626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:11.331905 20626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:11.332006 20626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:11.332101 20626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:47:11.338132 20626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.237.148:56441/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:36053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=34957
--tserver_master_addrs=127.31.237.190:40123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:11.339350 20626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:11.340729 20626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:11.351297 20637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:11.351657 20626 server_base.cc:1034] running on GCE node
W20260808 10:47:11.351908 20642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:11.352159 20636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:11.352676 20626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:11.353572 20626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:11.354777 20626 hybrid_clock.cc:648] HybridClock initialized: now 1786186031354733 us; error 54 us; skew 500 ppm
Aug 08 10:47:11 dist-test-slave-twwt krb5kdc[18646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186031, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:11.358503 20626 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:47:11.359741 20626 webserver.cc:460] Webserver started at http://127.31.237.131:34957/ using document root <none> and password file <none>
I20260808 10:47:11.360558 20626 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:11.360726 20626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:47:11 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186031, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:47:11.388286 20626 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 10:47:11.395526 20653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:11.397488 20626 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.000s
I20260808 10:47:11.397634 20626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "ae80d3b741fe42de9ac440198b85d556"
format_stamp: "Formatted at 2026-08-08 10:47:10 on dist-test-slave-twwt"
server_key: "0a2282a437c98ee4f9a3eabe86e5aa0d"
server_key_iv: "7c3e56610d2e2a99c1956e97c882af28"
server_key_version: "kuduclusterkey@0"
I20260808 10:47:11.398442 20626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:11.424369 20626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:11.428928 20626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:11.429232 20626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:11.430114 20626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:11.431397 20626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:11.431499 20626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:11.431573 20626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:11.431596 20626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:11.444114 20626 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:36053
I20260808 10:47:11.445618 20626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:47:11.446288 20831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:36053 every 8 connection(s)
I20260808 10:47:11.451320 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 20626
Aug 08 10:47:11 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186031, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:11.466104 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:11.450503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:38695 (local address 127.31.237.190:40123)
0808 10:47:11.450677 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:11.450682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:11.450932 (+ 250us) server_negotiation.cc:407] Connection header received
0808 10:47:11.452538 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:11.452542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:11.452602 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:11.452656 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:11.454819 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:11.455670 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:11.457403 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:11.457772 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:11.457901 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:11.461528 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:11.461532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:11.461535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:11.461565 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:11.463418 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:11.464106 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:11.464110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:11.464112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:11.464213 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:11.464742 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:11.464746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:11.464748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:11.464982 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:11.465074 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:11.465783 (+ 709us) server_negotiation.cc:299] Negotiation successful
0808 10:47:11.465927 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:47:11.467135 20834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:11.449341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:40123 (local address 127.31.237.131:38695)
0808 10:47:11.450759 (+ 1418us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:11.450817 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:11.452267 (+ 1450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:11.453443 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:11.453459 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:11.453830 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:11.454536 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:11.454556 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:11.455838 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:11.455841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:11.457235 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:11.457246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:11.458028 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:11.458032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:11.458232 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:11.459092 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:11.459113 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:11.461323 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:11.463553 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:11.463561 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:11.463579 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:11.463969 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:11.464362 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:11.464365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:11.464367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:11.464610 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:11.465132 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:11.465139 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:11.465469 (+ 330us) client_negotiation.cc:769] Sending connection context
0808 10:47:11.465678 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 10:47:11.466020 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1162,"thread_start_us":158,"threads_started":1}
I20260808 10:47:11.468758 20832 heartbeater.cc:347] Connected to a master server at 127.31.237.190:40123
I20260808 10:47:11.469039 20832 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:11.469604 20832 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
W20260808 10:47:11.471432 20832 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:40123 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:47:11.471557 20832 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:11.471719 20832 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
I20260808 10:47:11.472589 20832 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:11.472769 20832 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
W20260808 10:47:11.473541 20832 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:47:11.489805 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:11.453303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52252 (local address 127.31.237.190:40123)
0808 10:47:11.454907 (+ 1604us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:11.454911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:11.459812 (+ 4901us) server_negotiation.cc:407] Connection header received
0808 10:47:11.459872 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:11.459875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:11.459927 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:11.460012 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:11.461026 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:11.462187 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:11.464736 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:11.465082 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:11.465194 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:11.471935 (+ 6741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:11.471939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:11.471942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:11.471971 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:11.473837 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:11.474370 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:11.474374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:11.474376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:11.474469 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:11.478755 (+ 4286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:11.478760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:11.478761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:11.479052 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:11.479557 (+ 505us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:11.489476 (+ 9919us) server_negotiation.cc:299] Negotiation successful
0808 10:47:11.489650 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1517,"thread_start_us":76,"threads_started":1}
Aug 08 10:47:11 dist-test-slave-twwt krb5kdc[18646](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786186031, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:11.941716 20426 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:11.942085 20426 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
I20260808 10:47:11.943400 20139 ts_manager.cc:194] Registered new tserver with Master: 127fcec528074c9c8cb55eb7680e7221 (127.31.237.129:34163)
I20260808 10:47:11.945165 20139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:45163
I20260808 10:47:12.193828 20621 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:12.194171 20621 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
I20260808 10:47:12.195079 20139 ts_manager.cc:194] Registered new tserver with Master: 655a45d91fe548e1b15d506e74356b12 (127.31.237.130:34345)
I20260808 10:47:12.195961 20139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:36037
Aug 08 10:47:12 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186031, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:47:12.473793 20832 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:12.474124 20832 heartbeater.cc:511] Master 127.31.237.190:40123 requested a full tablet report, sending...
I20260808 10:47:12.474989 20139 ts_manager.cc:194] Registered new tserver with Master: ae80d3b741fe42de9ac440198b85d556 (127.31.237.131:36053)
I20260808 10:47:12.475812 20139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:38695
I20260808 10:47:12.479403 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:47:12.495977 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:12.481294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52364 (local address 127.31.237.190:40123)
0808 10:47:12.485031 (+ 3737us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:12.485036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:12.485053 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:47:12.485096 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:12.485099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:12.485147 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:12.485246 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:12.486244 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.487134 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.488351 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.488711 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.488819 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:12.490872 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:12.490877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:12.490880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:12.490908 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:12.493257 (+ 2349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:12.493791 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:12.493795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:12.493797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:12.493901 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:12.494233 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:12.494236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:12.494237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:12.494549 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:12.495564 (+ 1015us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:12.495656 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 10:47:12.495795 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3654,"thread_start_us":85,"threads_started":1}
I20260808 10:47:12.500648 20139 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:47:12.505489 20139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 08 10:47:12 dist-test-slave-twwt krb5kdc[18646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186031, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for HTTP/127.31.237.148@KRBTEST.COM
I20260808 10:47:12.547463 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:12.532107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:34345 (local address 127.0.0.1:52534)
0808 10:47:12.534514 (+ 2407us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:12.534530 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:12.534638 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:12.537889 (+ 3251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:12.537900 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:12.537938 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:12.538094 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:12.538101 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.544468 (+ 6367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.544471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:12.545576 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:12.545591 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.546871 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.546875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:12.546995 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:12.547041 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 10:47:12.547161 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 10:47:12.547287 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2337,"thread_start_us":90,"threads_started":1}
I20260808 10:47:12.547737 20878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:12.530449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:36053 (local address 127.0.0.1:50858)
0808 10:47:12.530940 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:12.530988 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:12.531103 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:12.543133 (+ 12030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:12.543137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:12.543158 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:12.543284 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:12.543292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.545006 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.545010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:12.546203 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:12.546215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.547307 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.547311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:12.547424 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:12.547451 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:47:12.547506 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 10:47:12.547594 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":122,"threads_started":1}
I20260808 10:47:12.548611 20879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:12.530565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50858 (local address 127.31.237.131:36053)
0808 10:47:12.542470 (+ 11905us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:12.542477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:12.542505 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:47:12.542592 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:12.542599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:12.542959 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:12.543457 (+ 498us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:12.543605 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.544861 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.546379 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.547154 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.548116 (+ 962us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:12.548247 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:12.548300 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 10:47:12.548440 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11768,"thread_start_us":93,"threads_started":1}
I20260808 10:47:12.549171 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:12.533156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:34163 (local address 127.0.0.1:41304)
0808 10:47:12.534800 (+ 1644us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:12.534811 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:12.534877 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:12.538221 (+ 3344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:12.538224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:12.538240 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:12.538316 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:12.538321 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.539872 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.539877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:12.541425 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:12.541450 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.548580 (+ 7130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.548584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:12.548671 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:12.548784 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 10:47:12.548979 (+ 195us) client_negotiation.cc:240] Negotiation successful
0808 10:47:12.549052 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1594,"thread_start_us":82,"threads_started":1}
I20260808 10:47:12.549752 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:12.533558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41304 (local address 127.31.237.129:34163)
0808 10:47:12.534196 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:12.534205 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:12.534962 (+ 757us) server_negotiation.cc:407] Connection header received
0808 10:47:12.535051 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:12.535057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:12.535277 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:12.535582 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:12.538436 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.539699 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.542270 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.543339 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.543487 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:12.543635 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:12.549499 (+ 5864us) server_negotiation.cc:299] Negotiation successful
0808 10:47:12.549618 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":76,"threads_started":1}
I20260808 10:47:12.552321 20735 tablet_service.cc:1467] Processing CreateTablet for tablet e33cb2dcb2b046489a00b98109d42ac5 (DEFAULT_TABLE table=test-table [id=38659e18b2864018a8a6c2d70235704c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:47:12.552961 20312 tablet_service.cc:1467] Processing CreateTablet for tablet e33cb2dcb2b046489a00b98109d42ac5 (DEFAULT_TABLE table=test-table [id=38659e18b2864018a8a6c2d70235704c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:47:12.553608 20735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e33cb2dcb2b046489a00b98109d42ac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:12.554081 20312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e33cb2dcb2b046489a00b98109d42ac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:12.563179 20889 tablet_bootstrap.cc:492] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556: Bootstrap starting.
I20260808 10:47:12.563551 20884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:12.532494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52534 (local address 127.31.237.130:34345)
0808 10:47:12.536191 (+ 3697us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:12.536198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:12.536221 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:47:12.536297 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:12.536303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:12.536506 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:12.536650 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:12.541603 (+ 4953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.544321 (+ 2718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.545730 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.546526 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.546664 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:12.546805 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:12.563237 (+ 16432us) server_negotiation.cc:299] Negotiation successful
0808 10:47:12.563396 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3415,"thread_start_us":64,"threads_started":1}
I20260808 10:47:12.566497 20889 tablet_bootstrap.cc:654] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:12.568621 20889 log.cc:824] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:12.569694 20527 tablet_service.cc:1467] Processing CreateTablet for tablet e33cb2dcb2b046489a00b98109d42ac5 (DEFAULT_TABLE table=test-table [id=38659e18b2864018a8a6c2d70235704c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:47:12.570977 20527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e33cb2dcb2b046489a00b98109d42ac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:12.571002 20890 tablet_bootstrap.cc:492] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221: Bootstrap starting.
I20260808 10:47:12.571794 20889 tablet_bootstrap.cc:492] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556: No bootstrap required, opened a new log
I20260808 10:47:12.572080 20889 ts_tablet_manager.cc:1397] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 10:47:12.573578 20890 tablet_bootstrap.cc:654] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:12.574545 20890 log.cc:824] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:12.576215 20889 raft_consensus.cc:348] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae80d3b741fe42de9ac440198b85d556" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 36053 } } peers { permanent_uuid: "655a45d91fe548e1b15d506e74356b12" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34345 } } peers { permanent_uuid: "127fcec528074c9c8cb55eb7680e7221" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34163 } }
I20260808 10:47:12.576613 20889 raft_consensus.cc:374] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:12.576716 20889 raft_consensus.cc:729] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae80d3b741fe42de9ac440198b85d556, State: Initialized, Role: FOLLOWER
I20260808 10:47:12.577301 20890 tablet_bootstrap.cc:492] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221: No bootstrap required, opened a new log
I20260808 10:47:12.577328 20889 consensus_queue.cc:260] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae80d3b741fe42de9ac440198b85d556" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 36053 } } peers { permanent_uuid: "655a45d91fe548e1b15d506e74356b12" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34345 } } peers { permanent_uuid: "127fcec528074c9c8cb55eb7680e7221" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34163 } }
I20260808 10:47:12.577512 20890 ts_tablet_manager.cc:1397] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 10:47:12.581182 20889 ts_tablet_manager.cc:1428] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260808 10:47:12.581502 20890 raft_consensus.cc:348] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae80d3b741fe42de9ac440198b85d556" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 36053 } } peers { permanent_uuid: "655a45d91fe548e1b15d506e74356b12" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34345 } } peers { permanent_uuid: "127fcec528074c9c8cb55eb7680e7221" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34163 } }
I20260808 10:47:12.581794 20890 raft_consensus.cc:374] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:12.581848 20890 raft_consensus.cc:729] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 127fcec528074c9c8cb55eb7680e7221, State: Initialized, Role: FOLLOWER
I20260808 10:47:12.582743 20890 consensus_queue.cc:260] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae80d3b741fe42de9ac440198b85d556" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 36053 } } peers { permanent_uuid: "655a45d91fe548e1b15d506e74356b12" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34345 } } peers { permanent_uuid: "127fcec528074c9c8cb55eb7680e7221" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34163 } }
I20260808 10:47:12.593459 20832 heartbeater.cc:503] Master 127.31.237.190:40123 was elected leader, sending a full tablet report...
I20260808 10:47:12.595351 20890 ts_tablet_manager.cc:1428] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260808 10:47:12.595829 20426 heartbeater.cc:503] Master 127.31.237.190:40123 was elected leader, sending a full tablet report...
I20260808 10:47:12.597618 20903 tablet_bootstrap.cc:492] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12: Bootstrap starting.
I20260808 10:47:12.601536 20903 tablet_bootstrap.cc:654] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:12.603013 20903 log.cc:824] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:12.606858 20903 tablet_bootstrap.cc:492] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12: No bootstrap required, opened a new log
I20260808 10:47:12.607197 20903 ts_tablet_manager.cc:1397] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20260808 10:47:12.616801 20903 raft_consensus.cc:348] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae80d3b741fe42de9ac440198b85d556" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 36053 } } peers { permanent_uuid: "655a45d91fe548e1b15d506e74356b12" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34345 } } peers { permanent_uuid: "127fcec528074c9c8cb55eb7680e7221" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34163 } }
I20260808 10:47:12.617358 20903 raft_consensus.cc:374] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:12.617473 20903 raft_consensus.cc:729] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 655a45d91fe548e1b15d506e74356b12, State: Initialized, Role: FOLLOWER
I20260808 10:47:12.618146 20899 raft_consensus.cc:482] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:47:12.618110 20903 consensus_queue.cc:260] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae80d3b741fe42de9ac440198b85d556" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 36053 } } peers { permanent_uuid: "655a45d91fe548e1b15d506e74356b12" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34345 } } peers { permanent_uuid: "127fcec528074c9c8cb55eb7680e7221" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34163 } }
I20260808 10:47:12.618386 20899 raft_consensus.cc:504] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae80d3b741fe42de9ac440198b85d556" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 36053 } } peers { permanent_uuid: "655a45d91fe548e1b15d506e74356b12" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34345 } } peers { permanent_uuid: "127fcec528074c9c8cb55eb7680e7221" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34163 } }
I20260808 10:47:12.618957 20621 heartbeater.cc:503] Master 127.31.237.190:40123 was elected leader, sending a full tablet report...
I20260808 10:47:12.620010 20899 leader_election.cc:288] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 655a45d91fe548e1b15d506e74356b12 (127.31.237.130:34345), 127fcec528074c9c8cb55eb7680e7221 (127.31.237.129:34163)
I20260808 10:47:12.621091 20903 ts_tablet_manager.cc:1428] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
I20260808 10:47:12.631640 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:12.621496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:41083 (local address 127.31.237.129:34163)
0808 10:47:12.621738 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:12.621743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:12.624434 (+ 2691us) server_negotiation.cc:407] Connection header received
0808 10:47:12.624780 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:12.624786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:12.624850 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:12.624902 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:12.627009 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.627715 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.630162 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.630788 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.630885 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:12.630942 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:12.631392 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 10:47:12.631489 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260808 10:47:12.632678 20913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:12.620902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:34163 (local address 127.31.237.131:41083)
0808 10:47:12.624341 (+ 3439us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:12.624359 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:12.624659 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:12.625996 (+ 1337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:12.626001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:12.626027 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:12.626257 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:12.626267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.628385 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.628388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:12.630013 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:12.630025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.631038 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.631041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:12.631216 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:12.631254 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 10:47:12.632046 (+ 792us) client_negotiation.cc:240] Negotiation successful
0808 10:47:12.632379 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3329,"thread_start_us":101,"threads_started":1}
I20260808 10:47:12.632910 20884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:12.621353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:48575 (local address 127.31.237.130:34345)
0808 10:47:12.621634 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:12.621642 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:12.622441 (+ 799us) server_negotiation.cc:407] Connection header received
0808 10:47:12.623202 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:12.623208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:12.623276 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:12.623333 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:12.625051 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.625835 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.630652 (+ 4817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.631267 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.631378 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:12.631425 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:12.632647 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0808 10:47:12.632782 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
I20260808 10:47:12.633262 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:12.620430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:34345 (local address 127.31.237.131:48575)
0808 10:47:12.622351 (+ 1921us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:12.622366 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:12.622488 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:12.623847 (+ 1359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:12.623850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:12.623873 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:12.624529 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:12.624543 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.626410 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.626415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:12.630484 (+ 4069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:12.630495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.632219 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.632222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:12.632454 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:12.632498 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 10:47:12.633064 (+ 566us) client_negotiation.cc:240] Negotiation successful
0808 10:47:12.633148 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1842,"thread_start_us":138,"threads_started":1}
I20260808 10:47:12.633715 20358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e33cb2dcb2b046489a00b98109d42ac5" candidate_uuid: "ae80d3b741fe42de9ac440198b85d556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "127fcec528074c9c8cb55eb7680e7221" is_pre_election: true
I20260808 10:47:12.634127 20358 raft_consensus.cc:2454] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae80d3b741fe42de9ac440198b85d556 in term 0.
I20260808 10:47:12.635106 20531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e33cb2dcb2b046489a00b98109d42ac5" candidate_uuid: "ae80d3b741fe42de9ac440198b85d556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655a45d91fe548e1b15d506e74356b12" is_pre_election: true
I20260808 10:47:12.635432 20531 raft_consensus.cc:2454] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae80d3b741fe42de9ac440198b85d556 in term 0.
I20260808 10:47:12.635938 20656 leader_election.cc:302] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 127fcec528074c9c8cb55eb7680e7221, ae80d3b741fe42de9ac440198b85d556; no voters:
I20260808 10:47:12.636274 20899 raft_consensus.cc:2781] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:47:12.636353 20899 raft_consensus.cc:482] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:47:12.636394 20899 raft_consensus.cc:3037] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:12.638021 20899 raft_consensus.cc:504] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae80d3b741fe42de9ac440198b85d556" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 36053 } } peers { permanent_uuid: "655a45d91fe548e1b15d506e74356b12" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34345 } } peers { permanent_uuid: "127fcec528074c9c8cb55eb7680e7221" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34163 } }
I20260808 10:47:12.638455 20899 leader_election.cc:288] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [CANDIDATE]: Term 1 election: Requested vote from peers 655a45d91fe548e1b15d506e74356b12 (127.31.237.130:34345), 127fcec528074c9c8cb55eb7680e7221 (127.31.237.129:34163)
I20260808 10:47:12.639945 20531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e33cb2dcb2b046489a00b98109d42ac5" candidate_uuid: "ae80d3b741fe42de9ac440198b85d556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "655a45d91fe548e1b15d506e74356b12"
I20260808 10:47:12.640087 20531 raft_consensus.cc:3037] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:12.641422 20531 raft_consensus.cc:2454] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae80d3b741fe42de9ac440198b85d556 in term 1.
I20260808 10:47:12.641835 20661 leader_election.cc:302] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 655a45d91fe548e1b15d506e74356b12, ae80d3b741fe42de9ac440198b85d556; no voters:
I20260808 10:47:12.642163 20358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e33cb2dcb2b046489a00b98109d42ac5" candidate_uuid: "ae80d3b741fe42de9ac440198b85d556" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "127fcec528074c9c8cb55eb7680e7221"
I20260808 10:47:12.642299 20358 raft_consensus.cc:3037] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:12.643715 20358 raft_consensus.cc:2454] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae80d3b741fe42de9ac440198b85d556 in term 1.
I20260808 10:47:12.644240 20899 raft_consensus.cc:2781] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:47:12.644495 20899 raft_consensus.cc:686] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [term 1 LEADER]: Becoming Leader. State: Replica: ae80d3b741fe42de9ac440198b85d556, State: Running, Role: LEADER
I20260808 10:47:12.644901 20899 consensus_queue.cc:237] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae80d3b741fe42de9ac440198b85d556" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 36053 } } peers { permanent_uuid: "655a45d91fe548e1b15d506e74356b12" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34345 } } peers { permanent_uuid: "127fcec528074c9c8cb55eb7680e7221" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34163 } }
I20260808 10:47:12.648566 20138 catalog_manager.cc:5495] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae80d3b741fe42de9ac440198b85d556 (127.31.237.131). New cstate: current_term: 1 leader_uuid: "ae80d3b741fe42de9ac440198b85d556" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae80d3b741fe42de9ac440198b85d556" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 36053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "655a45d91fe548e1b15d506e74356b12" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "127fcec528074c9c8cb55eb7680e7221" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34163 } health_report { overall_health: UNKNOWN } } }
W20260808 10:47:12.657845 20427 tablet.cc:2369] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:47:12.663538 20879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:12.658204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50860 (local address 127.31.237.131:36053)
0808 10:47:12.658358 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:12.658362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:12.658375 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:47:12.658452 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:12.658454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:12.658516 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:12.658644 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:47:12.659086 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.659884 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.661164 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:12.661512 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:12.661617 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:12.662067 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:47:12.662676 (+ 609us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:47:12.662805 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:12.663302 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 10:47:12.663365 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 10:47:12.670995 20358 raft_consensus.cc:1261] T e33cb2dcb2b046489a00b98109d42ac5 P 127fcec528074c9c8cb55eb7680e7221 [term 1 FOLLOWER]: Refusing update from remote peer ae80d3b741fe42de9ac440198b85d556: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:47:12.671113 20531 raft_consensus.cc:1261] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12 [term 1 FOLLOWER]: Refusing update from remote peer ae80d3b741fe42de9ac440198b85d556: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:47:12.671883 20914 consensus_queue.cc:1035] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [LEADER]: Connected to new peer: Peer: permanent_uuid: "655a45d91fe548e1b15d506e74356b12" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:47:12.672083 20899 consensus_queue.cc:1035] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [LEADER]: Connected to new peer: Peer: permanent_uuid: "127fcec528074c9c8cb55eb7680e7221" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:47:12.672565 20622 tablet.cc:2369] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:47:12.694751 20919 mvcc.cc:205] Tried to move back new op lower bound from 7316217989810634752 to 7316217989717135360. Current Snapshot: MvccSnapshot[applied={T|T < 7316217989810634752 or (T in {7316217989810634752})}]
I20260808 10:47:12.707757 20139 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:47:12.707963 20139 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:47:12.711447 20139 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5bee31ca13ad4cd2b29d3a06f2299fb6: Sending DeleteTablet for 3 replicas of tablet e33cb2dcb2b046489a00b98109d42ac5
I20260808 10:47:12.712965 20312 tablet_service.cc:1514] Processing DeleteTablet for tablet e33cb2dcb2b046489a00b98109d42ac5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:47:12 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:41304
I20260808 10:47:12.713579 20527 tablet_service.cc:1514] Processing DeleteTablet for tablet e33cb2dcb2b046489a00b98109d42ac5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:47:12 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:52534
I20260808 10:47:12.714733 20735 tablet_service.cc:1514] Processing DeleteTablet for tablet e33cb2dcb2b046489a00b98109d42ac5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:47:12 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:50858
I20260808 10:47:12.720575 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 20191
I20260808 10:47:12.733489 20928 tablet_replica.cc:330] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12: stopping tablet replica
I20260808 10:47:12.733883 20930 tablet_replica.cc:330] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556: stopping tablet replica
I20260808 10:47:12.739125 20930 raft_consensus.cc:2229] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:47:12.739689 20930 raft_consensus.cc:2258] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:47:12.741714 20930 ts_tablet_manager.cc:1905] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:47:12.743000 20928 raft_consensus.cc:2229] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:47:12.743482 20928 raft_consensus.cc:2258] T e33cb2dcb2b046489a00b98109d42ac5 P 655a45d91fe548e1b15d506e74356b12 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 10:47:12.744541 20120 connection.cc:535] server connection from 127.31.237.129:45163 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 10:47:12.744908 20118 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:34163 (error 108)
I20260808 10:47:12.746666 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:12.746266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:34163 (local address 127.0.0.1:41308)
0808 10:47:12.746465 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:12.746534 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:34163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260808 10:47:12.746974 20118 catalog_manager.cc:4584] TS 127fcec528074c9c8cb55eb7680e7221 (127.31.237.129:34163): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e33cb2dcb2b046489a00b98109d42ac5: Network error: Client connection negotiation failed: client connection to 127.31.237.129:34163: connect: Connection refused (error 111)
I20260808 10:47:12.747406 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 20428
I20260808 10:47:12.753842 20930 ts_tablet_manager.cc:1918] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:47:12.753986 20930 log.cc:1196] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/e33cb2dcb2b046489a00b98109d42ac5
I20260808 10:47:12.754413 20930 ts_tablet_manager.cc:1939] T e33cb2dcb2b046489a00b98109d42ac5 P ae80d3b741fe42de9ac440198b85d556: Deleting consensus metadata
I20260808 10:47:12.755985 20118 catalog_manager.cc:4841] TS ae80d3b741fe42de9ac440198b85d556 (127.31.237.131:36053): tablet e33cb2dcb2b046489a00b98109d42ac5 (table test-table [id=38659e18b2864018a8a6c2d70235704c]) successfully deleted
I20260808 10:47:12.761350 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 20626
I20260808 10:47:12.765671 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:12.765217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:34345 (local address 127.0.0.1:52550)
0808 10:47:12.765472 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:12.765550 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.130:34345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260808 10:47:12.775087 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 20090
I20260808 10:47:12.792775 32694 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 10:47:13.138844 32694 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 10:47:13.138980 32694 mini_ranger.cc:66] Stopping Ranger...
I20260808 10:47:13.484277 32694 mini_ranger.cc:68] Stopped Ranger
2026-08-08 10:47:13.484 UTC [18687] LOG: received smart shutdown request
2026-08-08 10:47:13.489 UTC [18687] LOG: background worker "logical replication launcher" (PID 18701) exited with exit code 1
2026-08-08 10:47:13.489 UTC [18692] LOG: shutting down
2026-08-08 10:47:13.639 UTC [18687] LOG: database system is shut down
2026-08-08T10:47:13Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51665 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:47:13 dist-test-slave-twwt krb5kdc[20944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:47:13 dist-test-slave-twwt krb5kdc[20944](info): set up 2 sockets
Aug 08 10:47:13 dist-test-slave-twwt krb5kdc[20944](info): commencing operation
krb5kdc: starting...
W20260808 10:47:15.027191 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:47:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:47:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:47:15.162144 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:38091
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:38137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:38091,127.31.237.189:38771,127.31.237.188:34935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:15.280932 20964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:15.281193 20964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:15.281232 20964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:15.285319 20964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:47:15.285385 20964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:15.285404 20964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:15.285421 20964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:47:15.285436 20964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:47:15.290686 20964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:38137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:38091,127.31.237.189:38771,127.31.237.188:34935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:38091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:15.291774 20964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:15.292833 20964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:15.302093 20973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:15.302093 20972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:15.302150 20975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:15.302525 20964 server_base.cc:1034] running on GCE node
I20260808 10:47:15.303167 20964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:15.304471 20964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:15.305693 20964 hybrid_clock.cc:648] HybridClock initialized: now 1786186035305651 us; error 54 us; skew 500 ppm
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:15.309002 20964 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:47:15.310082 20964 webserver.cc:460] Webserver started at http://127.31.237.190:39181/ using document root <none> and password file <none>
I20260808 10:47:15.310773 20964 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:15.310875 20964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:15.311133 20964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:47:15.313436 20964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "aa767139a80a47669864c2bad4bf28b4"
format_stamp: "Formatted at 2026-08-08 10:47:15 on dist-test-slave-twwt"
server_key: "a01a2ac4fbb6a809a156fee42325f52b"
server_key_iv: "58558a16e898bc0562ac2d0c8501c167"
server_key_version: "encryptionkey@0"
I20260808 10:47:15.314091 20964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "aa767139a80a47669864c2bad4bf28b4"
format_stamp: "Formatted at 2026-08-08 10:47:15 on dist-test-slave-twwt"
server_key: "a01a2ac4fbb6a809a156fee42325f52b"
server_key_iv: "58558a16e898bc0562ac2d0c8501c167"
server_key_version: "encryptionkey@0"
I20260808 10:47:15.318302 20964 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 10:47:15.321252 20986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:15.322544 20964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 10:47:15.322754 20964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "aa767139a80a47669864c2bad4bf28b4"
format_stamp: "Formatted at 2026-08-08 10:47:15 on dist-test-slave-twwt"
server_key: "a01a2ac4fbb6a809a156fee42325f52b"
server_key_iv: "58558a16e898bc0562ac2d0c8501c167"
server_key_version: "encryptionkey@0"
I20260808 10:47:15.322921 20964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:15.339768 20964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:15.342893 20964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:15.343102 20964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:15.350543 20964 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:38091
I20260808 10:47:15.350914 21064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:38091 every 8 connection(s)
I20260808 10:47:15.351596 20964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:47:15.354544 21065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:15.358538 21065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:47:15.359772 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 20964
I20260808 10:47:15.359956 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:47:15.360289 32694 external_mini_cluster.cc:1428] Setting key 8a3000eed19c82238b7cd4ce090fdf01
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.189@KRBTEST.COM" created.
I20260808 10:47:15.375016 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.361179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42016 (local address 127.31.237.190:38091)
0808 10:47:15.361658 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.361678 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.361965 (+ 287us) server_negotiation.cc:407] Connection header received
0808 10:47:15.363059 (+ 1094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.363064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.363245 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.363440 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.364300 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.365220 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.366241 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.366694 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.366875 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.369613 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.369618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.369629 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.369651 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.371559 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.372078 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.372086 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.372113 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.372227 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.372645 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.372652 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.372657 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.373193 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.373360 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.373789 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.374104 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":313,"thread_start_us":153,"threads_started":1}
I20260808 10:47:15.375022 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.361180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:38091 (local address 127.0.0.1:42016)
0808 10:47:15.361808 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.361847 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:15.362735 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:15.363440 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:15.363446 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:15.363905 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:15.364150 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.364164 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.365334 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.365337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.366118 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.366127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.366852 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.366856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.366967 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.367605 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:15.367622 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:15.369461 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:15.371676 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.371685 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.371694 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.371974 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.372360 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.372370 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.372372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.372498 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.373347 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:15.373354 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:15.373501 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 10:47:15.373672 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 10:47:15.373972 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":171,"threads_started":1}
I20260808 10:47:15.377928 21065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:47:15.378767 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.378463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:38771 (local address 127.0.0.1:37908)
0808 10:47:15.378590 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.378706 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:38771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 10:47:15.379077 20989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.237.189:38771: connect: Connection refused (error 111)
I20260808 10:47:15.380055 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.379848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:38771 (local address 127.0.0.1:37914)
0808 10:47:15.379955 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.379999 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:38771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 10:47:15.380412 21065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.189:38771: Network error: Client connection negotiation failed: client connection to 127.31.237.189:38771: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.189.keytab.
Entry for principal kudu/127.31.237.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.189.keytab.
I20260808 10:47:15.391811 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.189:38771
--webserver_interface=127.31.237.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.189.keytab
--principal=kudu/127.31.237.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:38137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:38091,127.31.237.189:38771,127.31.237.188:34935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:47:15.429695 21065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } attempt: 1
I20260808 10:47:15.430590 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.430283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:38771 (local address 127.0.0.1:37924)
0808 10:47:15.430436 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.430512 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:38771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 10:47:15.431421 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.431191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:38771 (local address 127.0.0.1:37934)
0808 10:47:15.431314 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.431362 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:38771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 10:47:15.431605 21065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.189:38771: Network error: Client connection negotiation failed: client connection to 127.31.237.189:38771: connect: Connection refused (error 111)
I20260808 10:47:15.499785 21065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } attempt: 2
I20260808 10:47:15.500640 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.500340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:38771 (local address 127.0.0.1:37948)
0808 10:47:15.500489 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.500563 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:38771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 10:47:15.501431 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.501195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:38771 (local address 127.0.0.1:37964)
0808 10:47:15.501329 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.501379 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:38771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 10:47:15.501555 21065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.189:38771: Network error: Client connection negotiation failed: client connection to 127.31.237.189:38771: connect: Connection refused (error 111)
W20260808 10:47:15.510362 21078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:15.510663 21078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:15.510764 21078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:15.514789 21078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:47:15.514860 21078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:15.514906 21078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:15.514941 21078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:47:15.514972 21078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:47:15.520278 21078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:38137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:38091,127.31.237.189:38771,127.31.237.188:34935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.189.keytab
--principal=kudu/127.31.237.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.189:38771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.237.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:15.521353 21078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:15.522390 21078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:15.531016 21088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:15.531016 21090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:15.531080 21092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:15.531347 21078 server_base.cc:1034] running on GCE node
I20260808 10:47:15.531877 21078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:15.532970 21078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:15.534157 21078 hybrid_clock.cc:648] HybridClock initialized: now 1786186035534141 us; error 24 us; skew 500 ppm
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:15.537385 21078 init.cc:377] Logged in from keytab as kudu/127.31.237.189@KRBTEST.COM (short username kudu)
I20260808 10:47:15.538292 21078 webserver.cc:460] Webserver started at http://127.31.237.189:38037/ using document root <none> and password file <none>
I20260808 10:47:15.538930 21078 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:15.539024 21078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:15.539261 21078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:47:15.541318 21078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/data/instance:
uuid: "aa61c380660b4ad2ad1621235fc0fcc7"
format_stamp: "Formatted at 2026-08-08 10:47:15 on dist-test-slave-twwt"
server_key: "f4594799bf7e0a63f5479eee02e14c10"
server_key_iv: "bb20d433851bd8ee9f62673f1c2cd802"
server_key_version: "encryptionkey@0"
I20260808 10:47:15.541899 21078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/wal/instance:
uuid: "aa61c380660b4ad2ad1621235fc0fcc7"
format_stamp: "Formatted at 2026-08-08 10:47:15 on dist-test-slave-twwt"
server_key: "f4594799bf7e0a63f5479eee02e14c10"
server_key_iv: "bb20d433851bd8ee9f62673f1c2cd802"
server_key_version: "encryptionkey@0"
I20260808 10:47:15.545588 21078 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.001s
I20260808 10:47:15.548156 21101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:15.549446 21078 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:47:15.549759 21078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/wal
uuid: "aa61c380660b4ad2ad1621235fc0fcc7"
format_stamp: "Formatted at 2026-08-08 10:47:15 on dist-test-slave-twwt"
server_key: "f4594799bf7e0a63f5479eee02e14c10"
server_key_iv: "bb20d433851bd8ee9f62673f1c2cd802"
server_key_version: "encryptionkey@0"
I20260808 10:47:15.549916 21078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:15.562800 21078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:15.566216 21078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:15.566423 21078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:15.573830 21078 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.189:38771
I20260808 10:47:15.574152 21191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.189:38771 every 8 connection(s)
I20260808 10:47:15.574891 21078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/data/info.pb
I20260808 10:47:15.577946 21193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:15.578902 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 21078
I20260808 10:47:15.579049 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-1/wal/instance
I20260808 10:47:15.579430 32694 external_mini_cluster.cc:1428] Setting key de736db395542049df6db4c428cb663a
I20260808 10:47:15.583052 21193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.237.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.188@KRBTEST.COM" created.
I20260808 10:47:15.592757 21065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } attempt: 3
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.189@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for kudu/127.31.237.189@KRBTEST.COM
I20260808 10:47:15.601194 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.585143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42030 (local address 127.31.237.190:38091)
0808 10:47:15.585318 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.585323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.586277 (+ 954us) server_negotiation.cc:407] Connection header received
0808 10:47:15.587411 (+ 1134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.587417 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.587500 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.587614 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.589074 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.589709 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.591138 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.591380 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.591472 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.595300 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.595306 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.595310 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.595343 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.597269 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.597984 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.597987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.597989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.598102 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.598666 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.598672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.598677 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.598956 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.599068 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.600794 (+ 1726us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.600961 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:47:15.601387 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.585562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:38091 (local address 127.0.0.1:42030)
0808 10:47:15.586089 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.586144 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:15.587168 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:15.587848 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:15.587855 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:15.588201 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:15.588861 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.588886 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.589894 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.589897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.591014 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.591025 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.591564 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.591568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.591780 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.592554 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:15.592573 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:15.595138 (+ 2565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:15.597428 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.597438 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.597460 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.597871 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.598215 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.598221 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.598226 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.598458 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.599131 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:15.599138 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:15.599513 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 10:47:15.599738 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 10:47:15.600138 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":150,"threads_started":1}
I20260808 10:47:15.603288 21193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:47:15.603868 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.593269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:38771 (local address 127.0.0.1:37970)
0808 10:47:15.593473 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.593490 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:15.593588 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:15.594325 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:15.594330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:15.594528 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:15.594710 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.594721 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.596022 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.596026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.596729 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.596739 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.597368 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.597373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.597486 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.597956 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:15.597974 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:15.599599 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:15.601883 (+ 2284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.601886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.601888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.602138 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.602717 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.602720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.602722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.602811 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.603319 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:15.603322 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:15.603427 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 10:47:15.603563 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 10:47:15.603704 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 10:47:15.604203 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.593412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37970 (local address 127.31.237.189:38771)
0808 10:47:15.593815 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.593829 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.593858 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:47:15.593911 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.593918 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.594144 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.594284 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.594845 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.595843 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.596857 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.597242 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.597366 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.599757 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.599766 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.599780 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.599803 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.601758 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.602451 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.602482 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.602487 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.602579 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.602922 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.602925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.602926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.603208 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.603355 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.603653 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.603794 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":157,"threads_started":1}
I20260808 10:47:15.606555 21065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:47:15.607295 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.607051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48086)
0808 10:47:15.607169 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.607229 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 10:47:15.608131 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.607883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48096)
0808 10:47:15.608029 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.608084 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260808 10:47:15.608343 21065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:34935: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.189@KRBTEST.COM for kudu/127.31.237.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.188.keytab.
Entry for principal kudu/127.31.237.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.188.keytab.
I20260808 10:47:15.613417 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.603821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:38771 (local address 127.0.0.1:37980)
0808 10:47:15.604179 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.604196 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:15.604276 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:15.604576 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:15.604580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:15.604714 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:15.604809 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.604814 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.605796 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.605799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.606424 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.606433 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.607165 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.607169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.607300 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.607823 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:15.607835 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:15.609246 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:15.611747 (+ 2501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.611752 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.611754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.612009 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.612298 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.612300 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.612302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.612386 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.612846 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:15.612849 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:15.612944 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 10:47:15.613051 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 10:47:15.613277 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":76}
I20260808 10:47:15.613432 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.604030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37980 (local address 127.31.237.189:38771)
0808 10:47:15.604348 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.604356 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.604367 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:47:15.604397 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.604399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.604432 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.604544 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.604949 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.605685 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.606563 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.607047 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.607169 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.609393 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.609397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.609400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.609425 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.611639 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.612110 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.612115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.612117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.612200 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.612480 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.612483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.612484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.612741 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.612858 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.613101 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.613225 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":259,"thread_start_us":129,"threads_started":1}
I20260808 10:47:15.614163 21193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:47:15.614562 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.188:34935
--webserver_interface=127.31.237.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.188.keytab
--principal=kudu/127.31.237.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:38137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:38091,127.31.237.189:38771,127.31.237.188:34935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:47:15.615031 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.614728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48104)
0808 10:47:15.614893 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.614970 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260808 10:47:15.615303 21106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
I20260808 10:47:15.616362 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.616106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48116)
0808 10:47:15.616243 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.616295 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 10:47:15.616580 21193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:34935: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
I20260808 10:47:15.639539 21065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } attempt: 1
I20260808 10:47:15.640484 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.640165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48128)
0808 10:47:15.640324 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.640398 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 10:47:15.641345 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.641118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48138)
0808 10:47:15.641234 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.641287 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 10:47:15.641522 21065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:34935: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
I20260808 10:47:15.665899 21193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } attempt: 1
I20260808 10:47:15.666852 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.666507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48148)
0808 10:47:15.666692 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.666765 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 10:47:15.667603 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.667376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48150)
0808 10:47:15.667489 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.667544 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 10:47:15.667737 21193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:34935: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
I20260808 10:47:15.716693 21065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } attempt: 2
I20260808 10:47:15.717650 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.717333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48162)
0808 10:47:15.717498 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.717565 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 10:47:15.718405 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.718190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48164)
0808 10:47:15.718300 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.718352 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 10:47:15.718533 21065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:34935: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
I20260808 10:47:15.735901 21193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } attempt: 2
W20260808 10:47:15.736356 21205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:15.736649 21205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:15.736752 21205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 10:47:15.736791 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.736447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48170)
0808 10:47:15.736591 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.736687 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 10:47:15.737514 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.737297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48174)
0808 10:47:15.737405 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.737452 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 10:47:15.737639 21193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:34935: Network error: Client connection negotiation failed: client connection to 127.31.237.188:34935: connect: Connection refused (error 111)
W20260808 10:47:15.741205 21205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:47:15.741300 21205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:15.741348 21205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:15.741406 21205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:47:15.741444 21205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:47:15.747190 21205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:38137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:38091,127.31.237.189:38771,127.31.237.188:34935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.188.keytab
--principal=kudu/127.31.237.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.188:34935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.237.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:15.748363 21205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:15.749516 21205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:15.758746 21218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:15.758828 21205 server_base.cc:1034] running on GCE node
W20260808 10:47:15.758756 21216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:15.758762 21215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:15.759708 21205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:15.760818 21205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:15.762001 21205 hybrid_clock.cc:648] HybridClock initialized: now 1786186035761988 us; error 28 us; skew 500 ppm
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:15.765144 21205 init.cc:377] Logged in from keytab as kudu/127.31.237.188@KRBTEST.COM (short username kudu)
I20260808 10:47:15.766191 21205 webserver.cc:460] Webserver started at http://127.31.237.188:36255/ using document root <none> and password file <none>
I20260808 10:47:15.766847 21205 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:15.766912 21205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:15.767175 21205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:47:15.769312 21205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/data/instance:
uuid: "552ab32bc4684318b9ba55a25f90ff16"
format_stamp: "Formatted at 2026-08-08 10:47:15 on dist-test-slave-twwt"
server_key: "3ec45996978895c8e5b8bd296642421e"
server_key_iv: "4f892aa10d14ebdc776d9f769badf015"
server_key_version: "encryptionkey@0"
I20260808 10:47:15.769909 21205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/wal/instance:
uuid: "552ab32bc4684318b9ba55a25f90ff16"
format_stamp: "Formatted at 2026-08-08 10:47:15 on dist-test-slave-twwt"
server_key: "3ec45996978895c8e5b8bd296642421e"
server_key_iv: "4f892aa10d14ebdc776d9f769badf015"
server_key_version: "encryptionkey@0"
I20260808 10:47:15.773932 21205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:47:15.776763 21228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:15.778159 21205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:47:15.778331 21205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/wal
uuid: "552ab32bc4684318b9ba55a25f90ff16"
format_stamp: "Formatted at 2026-08-08 10:47:15 on dist-test-slave-twwt"
server_key: "3ec45996978895c8e5b8bd296642421e"
server_key_iv: "4f892aa10d14ebdc776d9f769badf015"
server_key_version: "encryptionkey@0"
I20260808 10:47:15.778465 21205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:15.797627 21205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:15.801115 21205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:15.801342 21205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:15.809196 21205 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.188:34935
I20260808 10:47:15.809504 21301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.188:34935 every 8 connection(s)
I20260808 10:47:15.810276 21205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/data/info.pb
I20260808 10:47:15.813345 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 21205
I20260808 10:47:15.813438 21302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:15.813495 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786185862616645-32694-0/minicluster-data/master-2/wal/instance
I20260808 10:47:15.813804 32694 external_mini_cluster.cc:1428] Setting key 14ee73bcbda2bfe2cf9297034c686834
I20260808 10:47:15.817718 21065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } attempt: 3
I20260808 10:47:15.818959 21302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for kudu/127.31.237.188@KRBTEST.COM
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.188@KRBTEST.COM
I20260808 10:47:15.828585 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.818218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48200)
0808 10:47:15.818417 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.818433 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:15.818513 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:15.818915 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:15.818918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:15.819113 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:15.819214 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.819221 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.820246 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.820249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.821044 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.821058 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.821672 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.821679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.821788 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.822265 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:15.822286 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:15.823722 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:15.826563 (+ 2841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.826565 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.826567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.826901 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.827268 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.827271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.827273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.827384 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.828019 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:15.828023 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:15.828125 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 10:47:15.828244 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 10:47:15.828386 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.188@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:15.828783 21193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } attempt: 3
I20260808 10:47:15.830417 21310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.818306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48200 (local address 127.31.237.188:34935)
0808 10:47:15.818672 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.818682 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.818696 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:47:15.818730 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.818732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.818773 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.818866 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.819345 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.820122 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.821195 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.821534 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.822002 (+ 468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.823926 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.823931 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.823940 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.823993 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.826449 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.827016 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.827022 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.827028 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.827162 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.827469 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.827473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.827475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.827902 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.828053 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.829066 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.829363 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":275,"spinlock_wait_cycles":2944,"thread_start_us":135,"threads_started":1}
I20260808 10:47:15.831657 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.815233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.31.237.188:34935)
0808 10:47:15.815646 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.815663 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.815699 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:47:15.816394 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.816412 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.816755 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.817042 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.818041 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.819582 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.820499 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.821201 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.821805 (+ 604us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.824501 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.824505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.824507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.824533 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.826796 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.827400 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.827405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.827406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.827497 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.827894 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.827899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.827902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.828136 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.828310 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.828677 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.830050 (+ 1373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":731,"server-negotiator.queue_time_us":263,"thread_start_us":134,"threads_started":1}
I20260808 10:47:15.833914 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.821045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42034 (local address 127.31.237.190:38091)
0808 10:47:15.821154 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.821158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.821545 (+ 387us) server_negotiation.cc:407] Connection header received
0808 10:47:15.822004 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.822007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.822057 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.822135 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.823148 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.823934 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.825168 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.825459 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.825590 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.829667 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.829670 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.829674 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.829705 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.831448 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.832159 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.832164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.832168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.832270 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.832896 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.832900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.832902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.833155 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.833256 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.833652 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.833761 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:47:15.833931 21311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.820999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:38091 (local address 127.0.0.1:42034)
0808 10:47:15.821422 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.821474 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:15.821642 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:15.822319 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:15.822328 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:15.822851 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:15.823007 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.823015 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.824068 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.824073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.825034 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.825049 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.825585 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.825591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.825724 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.826750 (+ 1026us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:15.826768 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:15.829265 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:15.831734 (+ 2469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.831742 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.831747 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.832038 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.832391 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.832393 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.832395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.832487 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.833315 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:15.833321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:15.833432 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 10:47:15.833579 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 10:47:15.833775 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":113,"threads_started":1}
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.189@KRBTEST.COM for kudu/127.31.237.188@KRBTEST.COM
I20260808 10:47:15.835840 21302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:47:15.836007 21065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4: Bootstrap starting.
I20260808 10:47:15.838632 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.829283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48206)
0808 10:47:15.829426 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.829439 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:15.829534 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:15.830129 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:15.830132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:15.830315 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:15.830458 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.830467 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.831426 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.831430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.832138 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.832149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.832755 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.832759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.832864 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.833361 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:15.833377 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:15.835020 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:15.836984 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.836987 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.836990 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.837252 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.837553 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.837558 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.837561 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.837669 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.838095 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:15.838098 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:15.838201 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 10:47:15.838323 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 10:47:15.838457 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 10:47:15.839044 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.829582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.31.237.188:34935)
0808 10:47:15.829872 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.829881 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.829894 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:47:15.829928 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.829930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.829970 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.830594 (+ 624us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.830732 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.831298 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.832275 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.832597 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.833084 (+ 487us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.835158 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.835161 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.835164 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.835188 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.836881 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.837350 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.837354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.837355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.837442 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.837774 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.837777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.837779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.837984 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.838077 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.838761 (+ 684us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.838875 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":117,"threads_started":1}
I20260808 10:47:15.839170 21065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:15.840147 21065 log.cc:824] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.188@KRBTEST.COM for kudu/127.31.237.189@KRBTEST.COM
I20260808 10:47:15.842578 21193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7: Bootstrap starting.
I20260808 10:47:15.842944 21065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4: No bootstrap required, opened a new log
I20260808 10:47:15.845860 21193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:15.845888 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.836755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37984 (local address 127.31.237.189:38771)
0808 10:47:15.836878 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.836882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.836894 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:47:15.836953 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.836959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.837003 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.837086 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.837512 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.838248 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.839260 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.839601 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.839679 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.841998 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.842002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.842005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.842029 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.843714 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.844240 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.844309 (+ 69us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.844311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.844414 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.844879 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.844882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.844883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.845124 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.845197 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.845547 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.845667 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:47:15.846666 21311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.836582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:38771 (local address 127.0.0.1:37984)
0808 10:47:15.836674 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.836691 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:15.836774 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:15.837115 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:15.837118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:15.837273 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:15.837380 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.837388 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.838350 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.838356 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.839100 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.839114 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.839840 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.839844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.839925 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.840376 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:15.840392 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:15.841848 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:15.843842 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.843845 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.843847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.844116 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.844556 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.844559 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.844561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.844658 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.845274 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:15.845277 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:15.845368 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 10:47:15.846325 (+ 957us) client_negotiation.cc:240] Negotiation successful
0808 10:47:15.846497 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 10:47:15.846866 21193 log.cc:824] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:15.847623 21302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:47:15.847762 21065 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } }
I20260808 10:47:15.848029 21065 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:15.848100 21065 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa767139a80a47669864c2bad4bf28b4, State: Initialized, Role: FOLLOWER
I20260808 10:47:15.848687 21065 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } }
I20260808 10:47:15.849653 21317 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } } }
I20260808 10:47:15.849920 21317 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:47:15.850658 21065 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.188@KRBTEST.COM for kudu/127.31.237.188@KRBTEST.COM
I20260808 10:47:15.852596 21193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7: No bootstrap required, opened a new log
I20260808 10:47:15.852869 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.841131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42050 (local address 127.31.237.190:38091)
0808 10:47:15.841240 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.841244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.841258 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:47:15.841383 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.841387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.841425 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.841499 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.842598 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.843631 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.844820 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.845138 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.845238 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.848229 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.848232 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.848235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.848261 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.850088 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.850599 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.850649 (+ 50us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.850651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.850766 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.851351 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.851356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.851358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.851590 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.851799 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.852527 (+ 728us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.852659 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:47:15.856403 21311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.848131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:34935 (local address 127.0.0.1:48216)
0808 10:47:15.848273 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:15.848289 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:15.848363 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:15.848627 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:15.848630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:15.848809 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:15.848870 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.848875 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.849630 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.849633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.850321 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:15.850331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.850796 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.850799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:15.850883 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.851341 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:15.851354 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:15.852829 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:15.854963 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.854966 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.854968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.855190 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.855454 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:15.855457 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:15.855458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:15.855522 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:15.855896 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:15.855899 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:15.856029 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 10:47:15.856115 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:47:15.856246 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
W20260808 10:47:15.856873 21331 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 10:47:15.857086 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.848354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48216 (local address 127.31.237.188:34935)
0808 10:47:15.848451 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.848455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.848465 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:47:15.848491 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.848494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.848533 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.848617 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.848978 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.849517 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.850422 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.850683 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.851104 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.853007 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.853011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.853015 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.853040 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.854855 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.855305 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.855307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.855309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.855371 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.855608 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.855611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.855613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.855797 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.856779 (+ 982us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.856851 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.856960 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 10:47:15.857117 21331 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:47:15.857424 20964 master_runner.cc:186] Error getting master registration for 127.31.237.189:38771: OK, instance_id { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" instance_seqno: 1786186035571839 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:47:15.861274 21302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16: Bootstrap starting.
I20260808 10:47:15.861395 20964 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 10:47:15 dist-test-slave-twwt krb5kdc[20944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.189@KRBTEST.COM
I20260808 10:47:15.863938 21302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:15.864915 21302 log.cc:824] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:15.865061 21193 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } }
I20260808 10:47:15.865360 21193 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:15.865435 21193 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa61c380660b4ad2ad1621235fc0fcc7, State: Initialized, Role: FOLLOWER
I20260808 10:47:15.866058 21193 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } }
I20260808 10:47:15.866894 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.854799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37988 (local address 127.31.237.189:38771)
0808 10:47:15.854905 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.854909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.856079 (+ 1170us) server_negotiation.cc:407] Connection header received
0808 10:47:15.856257 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.856260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.856305 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.856374 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.857587 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.858325 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.859475 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.859781 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.859863 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.862996 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.863000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.863003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.863031 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.864828 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.865353 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.865359 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.865361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.865454 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.865787 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.865791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.865793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.866020 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.866326 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.866594 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.866745 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:47:15.867851 21193 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:47:15.867434 21334 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } } }
I20260808 10:47:15.868073 21334 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:47:15.870498 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:47:15.873028 21302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16: No bootstrap required, opened a new log
W20260808 10:47:15.876298 21348 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:47:15.877225 21348 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:47:15.878264 21302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } }
I20260808 10:47:15.878552 21302 raft_consensus.cc:374] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:15.882109 21302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 552ab32bc4684318b9ba55a25f90ff16, State: Initialized, Role: FOLLOWER
I20260808 10:47:15.882654 21302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } }
I20260808 10:47:15.884193 21302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:47:15.883431 21078 master_runner.cc:186] Error getting master registration for 127.31.237.188:34935: OK, instance_id { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" instance_seqno: 1786186035807049 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:47:15.887982 21078 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:47:15.890935 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.872899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42066 (local address 127.31.237.190:38091)
0808 10:47:15.873023 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.873028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.873888 (+ 860us) server_negotiation.cc:407] Connection header received
0808 10:47:15.874077 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.874080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.874129 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.874183 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.875240 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.876046 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.877336 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.877667 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.877741 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.881803 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.881809 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.881812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.881841 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.887417 (+ 5576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.889367 (+ 1950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.889371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.889373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.889481 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.889831 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.889837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.889842 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.890081 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.890218 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.890645 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.890777 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 10:47:15.891772 21376 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:47:15.891850 21376 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:47:15.892422 21205 master_runner.cc:263] No leader master found from master 552ab32bc4684318b9ba55a25f90ff16
I20260808 10:47:15.892503 21205 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:47:15.894791 21361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } } }
I20260808 10:47:15.895045 21361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:47:15.897586 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.876784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.31.237.189:38771)
0808 10:47:15.876903 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.876908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.881904 (+ 4996us) server_negotiation.cc:407] Connection header received
0808 10:47:15.882067 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.882070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.882519 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.882577 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.884383 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.885141 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.890712 (+ 5571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.891066 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.891144 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.893109 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.893114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.893117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.893145 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.895373 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.896157 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.896161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.896163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.896255 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.896572 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.896575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.896577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.896799 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.896915 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.897284 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.897425 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:47:15.898685 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:15.873508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48222 (local address 127.31.237.188:34935)
0808 10:47:15.873626 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:15.873630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:15.878408 (+ 4778us) server_negotiation.cc:407] Connection header received
0808 10:47:15.879698 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:15.879702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:15.879756 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:15.879808 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:15.880644 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.881329 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.891398 (+ 10069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:15.891711 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:15.891796 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:15.894728 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:15.894732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:15.894735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:15.894786 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:15.896454 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.896972 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.896975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.896977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.897068 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:15.897391 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:15.897394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:15.897396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:15.897613 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:15.897747 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:15.898371 (+ 624us) server_negotiation.cc:299] Negotiation successful
0808 10:47:15.898495 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:47:15.930830 21361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:47:15.931052 21361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } }
I20260808 10:47:15.932149 21361 leader_election.cc:288] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa767139a80a47669864c2bad4bf28b4 (127.31.237.190:38091), aa61c380660b4ad2ad1621235fc0fcc7 (127.31.237.189:38771)
I20260808 10:47:15.932974 21147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "552ab32bc4684318b9ba55a25f90ff16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" is_pre_election: true
I20260808 10:47:15.933372 21147 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 552ab32bc4684318b9ba55a25f90ff16 in term 0.
I20260808 10:47:15.933933 21233 leader_election.cc:302] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 552ab32bc4684318b9ba55a25f90ff16, aa61c380660b4ad2ad1621235fc0fcc7; no voters:
I20260808 10:47:15.934746 21024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "552ab32bc4684318b9ba55a25f90ff16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa767139a80a47669864c2bad4bf28b4" is_pre_election: true
I20260808 10:47:15.935058 21024 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 552ab32bc4684318b9ba55a25f90ff16 in term 0.
I20260808 10:47:15.935706 21361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:47:15.935796 21361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:47:15.935840 21361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:15.937237 21361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } }
I20260808 10:47:15.937937 21024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "552ab32bc4684318b9ba55a25f90ff16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa767139a80a47669864c2bad4bf28b4"
I20260808 10:47:15.938056 21024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:15.939569 21024 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 552ab32bc4684318b9ba55a25f90ff16 in term 1.
I20260808 10:47:15.939967 21235 leader_election.cc:302] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 552ab32bc4684318b9ba55a25f90ff16, aa767139a80a47669864c2bad4bf28b4; no voters:
I20260808 10:47:15.943229 21147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "552ab32bc4684318b9ba55a25f90ff16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa61c380660b4ad2ad1621235fc0fcc7"
I20260808 10:47:15.943377 21147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:15.944720 21147 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 552ab32bc4684318b9ba55a25f90ff16 in term 1.
I20260808 10:47:15.945278 21361 leader_election.cc:288] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [CANDIDATE]: Term 1 election: Requested vote from peers aa767139a80a47669864c2bad4bf28b4 (127.31.237.190:38091), aa61c380660b4ad2ad1621235fc0fcc7 (127.31.237.189:38771)
I20260808 10:47:15.945482 21361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:47:15.945605 21361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [term 1 LEADER]: Becoming Leader. State: Replica: 552ab32bc4684318b9ba55a25f90ff16, State: Running, Role: LEADER
I20260808 10:47:15.946012 21361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } }
I20260808 10:47:15.948024 21361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 552ab32bc4684318b9ba55a25f90ff16. Latest consensus state: current_term: 1 leader_uuid: "552ab32bc4684318b9ba55a25f90ff16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } } }
I20260808 10:47:15.948191 21361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:15.948670 21383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:47:15.949683 21383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:47:15.967489 21147 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [term 1 FOLLOWER]: Refusing update from remote peer 552ab32bc4684318b9ba55a25f90ff16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:47:15.969657 21380 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:47:15.974779 21334 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 552ab32bc4684318b9ba55a25f90ff16. Latest consensus state: current_term: 1 leader_uuid: "552ab32bc4684318b9ba55a25f90ff16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } } }
I20260808 10:47:15.974910 21334 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:47:15.976111 21024 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [term 1 FOLLOWER]: Refusing update from remote peer 552ab32bc4684318b9ba55a25f90ff16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:47:15.976783 21380 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:47:15.981060 21317 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 552ab32bc4684318b9ba55a25f90ff16. Latest consensus state: current_term: 1 leader_uuid: "552ab32bc4684318b9ba55a25f90ff16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } } }
I20260808 10:47:15.981185 21317 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:47:15.987643 21334 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "552ab32bc4684318b9ba55a25f90ff16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } } }
I20260808 10:47:15.987749 21334 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:47:15.988692 21380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "552ab32bc4684318b9ba55a25f90ff16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } } }
I20260808 10:47:15.988842 21380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:15.989197 21385 mvcc.cc:205] Tried to move back new op lower bound from 7316218003310657536 to 7316218003238707200. Current Snapshot: MvccSnapshot[applied={T|T < 7316218003310657536}]
I20260808 10:47:15.989414 21384 mvcc.cc:205] Tried to move back new op lower bound from 7316218003310657536 to 7316218003238707200. Current Snapshot: MvccSnapshot[applied={T|T < 7316218003310657536}]
I20260808 10:47:15.990967 21361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "552ab32bc4684318b9ba55a25f90ff16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } } }
I20260808 10:47:15.991071 21361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16 [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:15.992163 21317 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "552ab32bc4684318b9ba55a25f90ff16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa767139a80a47669864c2bad4bf28b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38091 } } peers { permanent_uuid: "aa61c380660b4ad2ad1621235fc0fcc7" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 38771 } } peers { permanent_uuid: "552ab32bc4684318b9ba55a25f90ff16" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 34935 } } }
I20260808 10:47:15.992287 21317 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:47:15.992805 21388 mvcc.cc:205] Tried to move back new op lower bound from 7316218003310657536 to 7316218003238707200. Current Snapshot: MvccSnapshot[applied={T|T < 7316218003310657536}]
I20260808 10:47:15.999583 21383 catalog_manager.cc:1347] Generated new cluster ID: 0c8e2e87686d4a8fa0c66ecd48c36198
I20260808 10:47:15.999642 21383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:47:16.029199 21383 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:47:16.033447 21383 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:47:16.049698 21383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 552ab32bc4684318b9ba55a25f90ff16: Generated new TSK 0
I20260808 10:47:16.050822 21383 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:47:16.858806 21331 catalog_manager.cc:1259] Loaded cluster ID: 0c8e2e87686d4a8fa0c66ecd48c36198
I20260808 10:47:16.858893 21331 catalog_manager.cc:1552] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4: loading cluster ID for follower catalog manager: success
I20260808 10:47:16.860549 21331 catalog_manager.cc:1574] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4: acquiring CA information for follower catalog manager: success
I20260808 10:47:16.861673 21331 catalog_manager.cc:1602] T 00000000000000000000000000000000 P aa767139a80a47669864c2bad4bf28b4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:47:16.878742 21348 catalog_manager.cc:1259] Loaded cluster ID: 0c8e2e87686d4a8fa0c66ecd48c36198
I20260808 10:47:16.878839 21348 catalog_manager.cc:1552] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7: loading cluster ID for follower catalog manager: success
I20260808 10:47:16.880457 21348 catalog_manager.cc:1574] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7: acquiring CA information for follower catalog manager: success
I20260808 10:47:16.881704 21348 catalog_manager.cc:1602] T 00000000000000000000000000000000 P aa61c380660b4ad2ad1621235fc0fcc7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:47:17.125030 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 20964
I20260808 10:47:17.140381 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 21078
I20260808 10:47:17.154297 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 21205
2026-08-08T10:47:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3399 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:47:17 dist-test-slave-twwt krb5kdc[21405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:47:17 dist-test-slave-twwt krb5kdc[21405](info): set up 2 sockets
Aug 08 10:47:17 dist-test-slave-twwt krb5kdc[21405](info): commencing operation
krb5kdc: starting...
W20260808 10:47:18.412102 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:47:18 dist-test-slave-twwt krb5kdc[21405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:47:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:47:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:47:18.545899 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:43555
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:43555
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:18.664793 21423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:18.665094 21423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:18.665171 21423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:18.669169 21423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:47:18.669274 21423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:18.669310 21423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:18.669375 21423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:47:18.669413 21423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:47:18.674823 21423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:43555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:43555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:18.675920 21423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:18.676988 21423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:18.685637 21429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:18.685644 21435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:18.685648 21433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:18.685976 21423 server_base.cc:1034] running on GCE node
I20260808 10:47:18.686947 21423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:18.688098 21423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:18.689301 21423 hybrid_clock.cc:648] HybridClock initialized: now 1786186038689289 us; error 31 us; skew 500 ppm
Aug 08 10:47:18 dist-test-slave-twwt krb5kdc[21405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186038, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:18.692272 21423 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:47:18.693289 21423 webserver.cc:460] Webserver started at http://127.31.237.190:36179/ using document root <none> and password file <none>
I20260808 10:47:18.693878 21423 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:18.693969 21423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:18.694208 21423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:47:18.696336 21423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "334a6fefe395483ab5d27b0ec30361cb"
format_stamp: "Formatted at 2026-08-08 10:47:18 on dist-test-slave-twwt"
server_key: "a2d5e36f03d893838782e816af8279a3"
server_key_iv: "0d6d02b34332f67e3ed21dbd171eb1e1"
server_key_version: "encryptionkey@0"
I20260808 10:47:18.696909 21423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "334a6fefe395483ab5d27b0ec30361cb"
format_stamp: "Formatted at 2026-08-08 10:47:18 on dist-test-slave-twwt"
server_key: "a2d5e36f03d893838782e816af8279a3"
server_key_iv: "0d6d02b34332f67e3ed21dbd171eb1e1"
server_key_version: "encryptionkey@0"
I20260808 10:47:18.700639 21423 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260808 10:47:18.703070 21450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:18.704382 21423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:47:18.704540 21423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "334a6fefe395483ab5d27b0ec30361cb"
format_stamp: "Formatted at 2026-08-08 10:47:18 on dist-test-slave-twwt"
server_key: "a2d5e36f03d893838782e816af8279a3"
server_key_iv: "0d6d02b34332f67e3ed21dbd171eb1e1"
server_key_version: "encryptionkey@0"
I20260808 10:47:18.704660 21423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:18.719653 21423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:18.723042 21423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:18.723332 21423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:18.730432 21423 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:43555
I20260808 10:47:18.730496 21521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:43555 every 8 connection(s)
I20260808 10:47:18.731479 21423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:47:18.733625 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 21423
I20260808 10:47:18.733738 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:47:18.734138 32694 external_mini_cluster.cc:1428] Setting key 88ffc94529f2b9a9ada8c23c85a85389
I20260808 10:47:18.734530 21522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:18.741619 21522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb: Bootstrap starting.
Aug 08 10:47:18 dist-test-slave-twwt krb5kdc[21405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:18.744426 21522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:18.745199 21522 log.cc:824] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:18.747216 21522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb: No bootstrap required, opened a new log
I20260808 10:47:18.749188 21526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:18.735535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56712 (local address 127.31.237.190:43555)
0808 10:47:18.735953 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:18.735970 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:18.736006 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:47:18.736657 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:18.736673 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:18.736992 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:18.737271 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:18.738212 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:18.739418 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:18.740627 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:18.741138 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:18.741258 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:18.744209 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:18.744215 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:18.744229 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:18.744259 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:18.746112 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:18.746731 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:18.746741 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:18.746746 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:18.746872 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:18.747181 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:18.747184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:18.747186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:18.747665 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:18.747925 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:18.748172 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 10:47:18.748473 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":141,"threads_started":1}
I20260808 10:47:18.751243 21522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "334a6fefe395483ab5d27b0ec30361cb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43555 } }
I20260808 10:47:18.751497 21522 raft_consensus.cc:374] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:18.751565 21522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 334a6fefe395483ab5d27b0ec30361cb, State: Initialized, Role: FOLLOWER
I20260808 10:47:18.752041 21522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "334a6fefe395483ab5d27b0ec30361cb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43555 } }
I20260808 10:47:18.752220 21522 raft_consensus.cc:388] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:47:18.752300 21522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:47:18.752310 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:47:18.752439 21522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:18.753521 21522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "334a6fefe395483ab5d27b0ec30361cb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43555 } }
I20260808 10:47:18.753893 21522 leader_election.cc:302] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 334a6fefe395483ab5d27b0ec30361cb; no voters:
I20260808 10:47:18.754261 21522 leader_election.cc:288] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:47:18.754405 21530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:47:18.754667 21530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [term 1 LEADER]: Becoming Leader. State: Replica: 334a6fefe395483ab5d27b0ec30361cb, State: Running, Role: LEADER
I20260808 10:47:18.755035 21530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "334a6fefe395483ab5d27b0ec30361cb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43555 } }
I20260808 10:47:18.755553 21522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:47:18.756808 21534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 334a6fefe395483ab5d27b0ec30361cb. Latest consensus state: current_term: 1 leader_uuid: "334a6fefe395483ab5d27b0ec30361cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "334a6fefe395483ab5d27b0ec30361cb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43555 } } }
I20260808 10:47:18.756861 21532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "334a6fefe395483ab5d27b0ec30361cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "334a6fefe395483ab5d27b0ec30361cb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43555 } } }
I20260808 10:47:18.757001 21532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:18.757282 21534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:18.757498 21545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:47:18.760404 21545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:47:18.766701 21545 catalog_manager.cc:1347] Generated new cluster ID: 388524843f9b4fb2a9516de94dcc2b16
I20260808 10:47:18.766780 21545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:47:18.786810 21545 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:47:18.788198 21545 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:47:18.803706 21545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 334a6fefe395483ab5d27b0ec30361cb: Generated new TSK 0
I20260808 10:47:18.804672 21545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:47:18.859756 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 21423
2026-08-08T10:47:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1709 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:47:18 dist-test-slave-twwt krb5kdc[21564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:47:18 dist-test-slave-twwt krb5kdc[21564](info): set up 2 sockets
Aug 08 10:47:18 dist-test-slave-twwt krb5kdc[21564](info): commencing operation
krb5kdc: starting...
W20260808 10:47:20.118788 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:47:20 dist-test-slave-twwt krb5kdc[21564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:47:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:47:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:47:20.249708 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:34147
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:34147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:20.365790 21583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:20.366106 21583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:20.366179 21583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:20.370213 21583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:47:20.370316 21583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:20.370348 21583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:20.370376 21583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:47:20.370410 21583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:47:20.375788 21583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:34147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:34147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:20.377017 21583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:20.378108 21583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:20.386837 21589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:20.386839 21592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:20.387008 21583 server_base.cc:1034] running on GCE node
W20260808 10:47:20.387209 21590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:20.387800 21583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:20.388938 21583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:20.390169 21583 hybrid_clock.cc:648] HybridClock initialized: now 1786186040390138 us; error 37 us; skew 500 ppm
Aug 08 10:47:20 dist-test-slave-twwt krb5kdc[21564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186040, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:20.393384 21583 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:47:20.394438 21583 webserver.cc:460] Webserver started at http://127.31.237.190:35951/ using document root <none> and password file <none>
I20260808 10:47:20.395104 21583 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:20.395174 21583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:20.395363 21583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:47:20.397666 21583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "0e53e08be91e402aba993003729aca96"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "4e50087ef5cae52dfb4378f3e11d7456"
server_key_iv: "e34f8b61c7ffa7ff8d9a30f0d005218a"
server_key_version: "encryptionkey@0"
I20260808 10:47:20.398257 21583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "0e53e08be91e402aba993003729aca96"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "4e50087ef5cae52dfb4378f3e11d7456"
server_key_iv: "e34f8b61c7ffa7ff8d9a30f0d005218a"
server_key_version: "encryptionkey@0"
I20260808 10:47:20.402427 21583 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 10:47:20.405220 21601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:20.406558 21583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:47:20.406750 21583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "0e53e08be91e402aba993003729aca96"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "4e50087ef5cae52dfb4378f3e11d7456"
server_key_iv: "e34f8b61c7ffa7ff8d9a30f0d005218a"
server_key_version: "encryptionkey@0"
I20260808 10:47:20.406870 21583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:20.431505 21583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:20.434576 21583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:20.434813 21583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:20.442292 21583 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:34147
I20260808 10:47:20.442582 21676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:34147 every 8 connection(s)
I20260808 10:47:20.443490 21583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:47:20.446403 21678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:20.447253 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 21583
I20260808 10:47:20.447409 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:47:20.447731 32694 external_mini_cluster.cc:1428] Setting key 647a2254dfe0cf07d16952d9cb375e7c
I20260808 10:47:20.453126 21678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Bootstrap starting.
I20260808 10:47:20.456012 21678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Neither blocks nor log segments found. Creating new log.
Aug 08 10:47:20 dist-test-slave-twwt krb5kdc[21564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:20.456885 21678 log.cc:824] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:20.459393 21678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: No bootstrap required, opened a new log
I20260808 10:47:20.463006 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:20.449142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39400 (local address 127.31.237.190:34147)
0808 10:47:20.449585 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:20.449601 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:20.449639 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:47:20.450276 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:20.450292 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:20.450694 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:20.450981 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:20.451879 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:20.452872 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:20.453996 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:20.454499 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:20.454693 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:20.457539 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:20.457544 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:20.457557 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:20.457584 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:20.459761 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:20.460203 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:20.460209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:20.460216 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:20.460372 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:20.460641 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:20.460644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:20.460646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:20.461161 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:20.461377 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:20.461598 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 10:47:20.461974 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":136,"threads_started":1}
I20260808 10:47:20.464210 21678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } }
I20260808 10:47:20.464457 21678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:20.464520 21678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e53e08be91e402aba993003729aca96, State: Initialized, Role: FOLLOWER
I20260808 10:47:20.464991 21678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } }
I20260808 10:47:20.465158 21678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:47:20.465270 21678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:47:20.465375 21678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:20.466274 21678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } }
I20260808 10:47:20.466572 21678 leader_election.cc:302] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e53e08be91e402aba993003729aca96; no voters:
I20260808 10:47:20.466907 21678 leader_election.cc:288] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:47:20.467036 21686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:47:20.467314 21686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 1 LEADER]: Becoming Leader. State: Replica: 0e53e08be91e402aba993003729aca96, State: Running, Role: LEADER
I20260808 10:47:20.467651 21686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } }
I20260808 10:47:20.468303 21678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:47:20.469429 21688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e53e08be91e402aba993003729aca96. Latest consensus state: current_term: 1 leader_uuid: "0e53e08be91e402aba993003729aca96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } } }
I20260808 10:47:20.469560 21688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:20.469854 21687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e53e08be91e402aba993003729aca96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } } }
I20260808 10:47:20.469938 21687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:20.469964 21700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:47:20.472687 21700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:47:20.478698 21700 catalog_manager.cc:1347] Generated new cluster ID: 12232d378e5541e395c50311b32274b8
I20260808 10:47:20.478765 21700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:47:20.491940 21700 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:47:20.493206 21700 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:47:20.501195 21700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Generated new TSK 0
I20260808 10:47:20.502031 21700 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:47:20.543789 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:34147
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:20.667094 21720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:20.667387 21720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:20.667431 21720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:20.671777 21720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:20.671841 21720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:20.671903 21720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:47:20.677277 21720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:20.678328 21720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:20.679371 21720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:20.689675 21730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:20.689675 21726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:20.689819 21728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:20.690073 21720 server_base.cc:1034] running on GCE node
I20260808 10:47:20.690836 21720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:20.691555 21720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:20.692746 21720 hybrid_clock.cc:648] HybridClock initialized: now 1786186040692716 us; error 45 us; skew 500 ppm
Aug 08 10:47:20 dist-test-slave-twwt krb5kdc[21564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186040, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:20.696126 21720 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:47:20.697183 21720 webserver.cc:460] Webserver started at http://127.31.237.129:43729/ using document root <none> and password file <none>
I20260808 10:47:20.697885 21720 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:20.697988 21720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:20.698235 21720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:47:20.700502 21720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "c53d5237a75840849a3ce6d7f10445d6"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "3c8e45c0e5552c52e3f1eca8da8242db"
server_key_iv: "2124f5265c2f0d1b19e5ef08c35017d8"
server_key_version: "encryptionkey@0"
I20260808 10:47:20.701121 21720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "c53d5237a75840849a3ce6d7f10445d6"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "3c8e45c0e5552c52e3f1eca8da8242db"
server_key_iv: "2124f5265c2f0d1b19e5ef08c35017d8"
server_key_version: "encryptionkey@0"
I20260808 10:47:20.705276 21720 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:47:20.708250 21743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:20.709693 21720 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:47:20.709851 21720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "c53d5237a75840849a3ce6d7f10445d6"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "3c8e45c0e5552c52e3f1eca8da8242db"
server_key_iv: "2124f5265c2f0d1b19e5ef08c35017d8"
server_key_version: "encryptionkey@0"
I20260808 10:47:20.709987 21720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:20.727062 21720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:20.730993 21720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:20.731228 21720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:20.731920 21720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:20.732879 21720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:20.732957 21720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:20.733036 21720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:20.733085 21720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:20.743232 21720 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:37879
I20260808 10:47:20.743321 21912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:37879 every 8 connection(s)
I20260808 10:47:20.744350 21720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:47:20.751952 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 21720
I20260808 10:47:20.752152 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:47:20.752497 32694 external_mini_cluster.cc:1428] Setting key 16a46feacf7f0678c9dbc682f0a868f1
Aug 08 10:47:20 dist-test-slave-twwt krb5kdc[21564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186040, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:20.760360 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:20.746506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:35145 (local address 127.31.237.190:34147)
0808 10:47:20.746771 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:20.746776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:20.747715 (+ 939us) server_negotiation.cc:407] Connection header received
0808 10:47:20.749303 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:20.749307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:20.749372 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:20.749483 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:20.750804 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:20.751406 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:20.753147 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:20.753419 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:20.753546 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:20.756386 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:20.756390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:20.756392 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:20.756420 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:20.758022 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:20.758624 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:20.758630 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:20.758633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:20.758727 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:20.759122 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:20.759125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:20.759127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:20.759338 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:20.759453 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:20.760030 (+ 577us) server_negotiation.cc:299] Negotiation successful
0808 10:47:20.760172 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260808 10:47:20.761279 21915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:20.746978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:35145)
0808 10:47:20.747518 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:20.747586 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:20.749041 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:20.749687 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:20.749694 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:20.750091 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:20.750637 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:20.750654 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:20.751832 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:20.751836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:20.752997 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:20.753009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:20.753594 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:20.753598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:20.753796 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:20.754478 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:20.754497 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:20.756181 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:20.758161 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:20.758168 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:20.758188 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:20.758479 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:20.758833 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:20.758837 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:20.758839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:20.759002 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:20.759455 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:20.759465 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:20.759758 (+ 293us) client_negotiation.cc:769] Sending connection context
0808 10:47:20.759972 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 10:47:20.760352 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":142,"threads_started":1}
I20260808 10:47:20.763129 21913 heartbeater.cc:347] Connected to a master server at 127.31.237.190:34147
I20260808 10:47:20.763422 21913 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:47:20.763999 21913 heartbeater.cc:511] Master 127.31.237.190:34147 requested a full tablet report, sending...
I20260808 10:47:20.765884 21629 ts_manager.cc:194] Registered new tserver with Master: c53d5237a75840849a3ce6d7f10445d6 (127.31.237.129:37879)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
I20260808 10:47:20.767396 21629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:35145
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:47:20.787212 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:34147
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:20.905264 21919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:20.905521 21919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:20.905565 21919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:20.909462 21919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:20.909519 21919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:20.909576 21919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:47:20.915345 21919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:20.916363 21919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:20.917361 21919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:20.927214 21928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:20.927218 21925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:20.927248 21930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:20.927277 21919 server_base.cc:1034] running on GCE node
I20260808 10:47:20.927899 21919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:20.928531 21919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:20.929703 21919 hybrid_clock.cc:648] HybridClock initialized: now 1786186040929668 us; error 46 us; skew 500 ppm
Aug 08 10:47:20 dist-test-slave-twwt krb5kdc[21564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186040, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:20.932637 21919 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:47:20.933625 21919 webserver.cc:460] Webserver started at http://127.31.237.130:38361/ using document root <none> and password file <none>
I20260808 10:47:20.934207 21919 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:20.934269 21919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:20.934492 21919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:47:20.936599 21919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "59a4550bf97a4de6a4c1ea827be7e81f"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "7a2a147f820e869dc60dd50ba8006685"
server_key_iv: "5b6b784ee0c5a90ca50766dd36d4ad24"
server_key_version: "encryptionkey@0"
I20260808 10:47:20.937211 21919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "59a4550bf97a4de6a4c1ea827be7e81f"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "7a2a147f820e869dc60dd50ba8006685"
server_key_iv: "5b6b784ee0c5a90ca50766dd36d4ad24"
server_key_version: "encryptionkey@0"
I20260808 10:47:20.941054 21919 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 10:47:20.943594 21940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:20.944849 21919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:47:20.944993 21919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "59a4550bf97a4de6a4c1ea827be7e81f"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "7a2a147f820e869dc60dd50ba8006685"
server_key_iv: "5b6b784ee0c5a90ca50766dd36d4ad24"
server_key_version: "encryptionkey@0"
I20260808 10:47:20.945111 21919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:20.961395 21919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:20.964645 21919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:20.964874 21919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:20.965476 21919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:20.966328 21919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:20.966400 21919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:20.966473 21919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:20.966526 21919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:20.976591 21919 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:34977
I20260808 10:47:20.976612 22105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:34977 every 8 connection(s)
I20260808 10:47:20.977634 21919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:47:20.985071 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 21919
I20260808 10:47:20.985236 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:47:20.985555 32694 external_mini_cluster.cc:1428] Setting key 50003e55a824acb7ec27ff21822a4caf
Aug 08 10:47:20 dist-test-slave-twwt krb5kdc[21564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186040, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:20.993432 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:20.979754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:40885 (local address 127.31.237.190:34147)
0808 10:47:20.979948 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:20.979952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:20.980919 (+ 967us) server_negotiation.cc:407] Connection header received
0808 10:47:20.982409 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:20.982413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:20.982473 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:20.982582 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:20.983903 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:20.984437 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:20.985762 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:20.986075 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:20.986203 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:20.989097 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:20.989104 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:20.989110 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:20.989146 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:20.990792 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:20.991469 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:20.991473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:20.991474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:20.991555 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:20.991978 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:20.991981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:20.991982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:20.992169 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:20.992251 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:20.993059 (+ 808us) server_negotiation.cc:299] Negotiation successful
0808 10:47:20.993261 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260808 10:47:20.994725 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:20.980207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:40885)
0808 10:47:20.980727 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:20.980775 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:20.982144 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:20.982803 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:20.982810 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:20.983200 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:20.983736 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:20.983753 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:20.984576 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:20.984580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:20.985607 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:20.985617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:20.986248 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:20.986251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:20.986418 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:20.987162 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:20.987180 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:20.988914 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:20.990916 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:20.990925 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:20.990945 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:20.991364 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:20.991653 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:20.991659 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:20.991664 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:20.991876 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:20.992281 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:20.992290 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:20.992672 (+ 382us) client_negotiation.cc:769] Sending connection context
0808 10:47:20.992993 (+ 321us) client_negotiation.cc:240] Negotiation successful
0808 10:47:20.993428 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":128,"threads_started":1}
I20260808 10:47:20.996101 22106 heartbeater.cc:347] Connected to a master server at 127.31.237.190:34147
I20260808 10:47:20.996403 22106 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:20.996954 22106 heartbeater.cc:511] Master 127.31.237.190:34147 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:47:20.998214 21629 ts_manager.cc:194] Registered new tserver with Master: 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977)
I20260808 10:47:20.998957 21629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:40885
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:47:21.018211 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:34147
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:21.136976 22112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:21.137266 22112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:21.137311 22112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:21.141427 22112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:21.141498 22112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:21.141602 22112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:47:21.147230 22112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:21.148461 22112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:21.149639 22112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:21.159893 22122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:21.159893 22119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:21.159842 22120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:21.160166 22112 server_base.cc:1034] running on GCE node
I20260808 10:47:21.160640 22112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:21.161309 22112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:21.162489 22112 hybrid_clock.cc:648] HybridClock initialized: now 1786186041162472 us; error 29 us; skew 500 ppm
Aug 08 10:47:21 dist-test-slave-twwt krb5kdc[21564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186041, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:21.165678 22112 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:47:21.166788 22112 webserver.cc:460] Webserver started at http://127.31.237.131:46323/ using document root <none> and password file <none>
I20260808 10:47:21.167403 22112 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:21.167505 22112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:21.167750 22112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:47:21.170142 22112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "d709d62000ff4579a6bf93923e440cdc"
format_stamp: "Formatted at 2026-08-08 10:47:21 on dist-test-slave-twwt"
server_key: "ad1142777267a1b3b6eb7cba12810e0e"
server_key_iv: "2924571fcf10922815afeb0f24f395ba"
server_key_version: "encryptionkey@0"
I20260808 10:47:21.170817 22112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "d709d62000ff4579a6bf93923e440cdc"
format_stamp: "Formatted at 2026-08-08 10:47:21 on dist-test-slave-twwt"
server_key: "ad1142777267a1b3b6eb7cba12810e0e"
server_key_iv: "2924571fcf10922815afeb0f24f395ba"
server_key_version: "encryptionkey@0"
I20260808 10:47:21.174872 22112 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:47:21.177446 22132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:21.179024 22112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:47:21.179171 22112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "d709d62000ff4579a6bf93923e440cdc"
format_stamp: "Formatted at 2026-08-08 10:47:21 on dist-test-slave-twwt"
server_key: "ad1142777267a1b3b6eb7cba12810e0e"
server_key_iv: "2924571fcf10922815afeb0f24f395ba"
server_key_version: "encryptionkey@0"
I20260808 10:47:21.179296 22112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:21.200659 22112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:21.204420 22112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:21.204660 22112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:21.205338 22112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:21.206346 22112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:21.206426 22112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:21.206504 22112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:21.206560 22112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:21.216579 22112 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:39973
I20260808 10:47:21.216635 22293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:39973 every 8 connection(s)
I20260808 10:47:21.217612 22112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:47:21.226739 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 22112
I20260808 10:47:21.226912 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:47:21.227232 32694 external_mini_cluster.cc:1428] Setting key 873b685d584d8b999cc1569038ab2424
Aug 08 10:47:21 dist-test-slave-twwt krb5kdc[21564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186041, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:21.232450 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.219575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:47713 (local address 127.31.237.190:34147)
0808 10:47:21.219724 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:21.219728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:21.220627 (+ 899us) server_negotiation.cc:407] Connection header received
0808 10:47:21.222063 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:21.222067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:21.222128 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:21.222211 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:21.223542 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.224084 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.225275 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.225504 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.225619 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:21.228610 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:21.228615 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:21.228618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:21.228649 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:21.230284 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:21.230991 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:21.230994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:21.230995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:21.231071 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:21.231416 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:21.231418 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:21.231419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:21.231591 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:21.231664 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:21.232199 (+ 535us) server_negotiation.cc:299] Negotiation successful
0808 10:47:21.232323 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:47:21.233395 22296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.220010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:47713)
0808 10:47:21.220474 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:21.220517 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:21.221829 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:21.222395 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:21.222402 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:21.222874 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:21.223379 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:21.223393 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.224241 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.224250 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:21.225138 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:21.225148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.225674 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.225678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:21.225845 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:21.226449 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:21.226465 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:21.228398 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:21.230439 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:21.230445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:21.230459 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:21.230883 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:21.231161 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:21.231163 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:21.231164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:21.231322 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:21.231690 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:21.231695 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:21.231965 (+ 270us) client_negotiation.cc:769] Sending connection context
0808 10:47:21.232134 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 10:47:21.232441 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":105,"threads_started":1}
I20260808 10:47:21.234557 22294 heartbeater.cc:347] Connected to a master server at 127.31.237.190:34147
I20260808 10:47:21.234869 22294 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:21.235411 22294 heartbeater.cc:511] Master 127.31.237.190:34147 requested a full tablet report, sending...
I20260808 10:47:21.236521 21629 ts_manager.cc:194] Registered new tserver with Master: d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973)
I20260808 10:47:21.237142 21629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:47713
I20260808 10:47:21.241460 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:47:21.251423 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.243527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.31.237.190:34147)
0808 10:47:21.243652 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:21.243656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:21.243880 (+ 224us) server_negotiation.cc:407] Connection header received
0808 10:47:21.243947 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:21.243949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:21.243991 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:21.244059 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:21.244805 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.245372 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.246304 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.246537 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.246718 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:21.248118 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:21.248121 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:21.248123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:21.248147 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:21.249833 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:21.250258 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:21.250261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:21.250263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:21.250357 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:21.250601 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:21.250620 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:21.250622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:21.250846 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:21.250960 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:21.251185 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 10:47:21.251284 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:47:21.254385 21629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:47:21.257577 21629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:47:21.273506 22308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.267031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:37879 (local address 127.0.0.1:43076)
0808 10:47:21.267906 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:21.267964 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:21.268153 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:21.268936 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:21.268943 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:21.268966 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:21.269148 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:21.269156 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.270503 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.270507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:21.271901 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:21.271913 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.272932 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.272935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:21.273039 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:21.273075 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 10:47:21.273213 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 10:47:21.273353 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":46,"thread_start_us":86,"threads_started":1}
I20260808 10:47:21.273514 22312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.267357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.31.237.129:37879)
0808 10:47:21.267646 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:21.267654 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:21.268078 (+ 424us) server_negotiation.cc:407] Connection header received
0808 10:47:21.268326 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:21.268335 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:21.268590 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:21.268751 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:21.269376 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.270387 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.272038 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.272796 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.272945 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:21.273093 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:21.273268 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 10:47:21.273362 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":103,"threads_started":1}
I20260808 10:47:21.273908 22309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.267139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:34977 (local address 127.0.0.1:54816)
0808 10:47:21.268206 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:21.268219 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:21.268313 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:21.269312 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:21.269315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:21.269336 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:21.269460 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:21.269467 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.271025 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.271029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:21.272206 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:21.272214 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.273343 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.273346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:21.273441 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:21.273463 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:47:21.273511 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 10:47:21.273729 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"mutex_wait_us":49,"thread_start_us":108,"threads_started":1}
I20260808 10:47:21.274106 22315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.267981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:39973 (local address 127.0.0.1:50264)
0808 10:47:21.268505 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:21.268517 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:21.268603 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:21.269536 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:21.269539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:21.269557 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:21.269657 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:21.269663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.271061 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.271063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:21.272393 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:21.272404 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.273637 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.273642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:21.273758 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:21.273780 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:47:21.273850 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 10:47:21.273909 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":246,"thread_start_us":120,"threads_started":1}
I20260808 10:47:21.274765 22313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.267736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54816 (local address 127.31.237.130:34977)
0808 10:47:21.268851 (+ 1115us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:21.268876 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:21.268900 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:47:21.268968 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:21.268973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:21.269152 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:21.269339 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:21.269600 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.270855 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.272337 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.273191 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.274281 (+ 1090us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:21.274426 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:21.274477 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 10:47:21.274570 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1006,"thread_start_us":192,"threads_started":1}
I20260808 10:47:21.274842 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.268560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50264 (local address 127.31.237.131:39973)
0808 10:47:21.268950 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:21.268959 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:21.268978 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:47:21.269042 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:21.269045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:21.269285 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:21.269449 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:21.269791 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.270926 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.272523 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.273330 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.273437 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:21.273568 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:21.273997 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 10:47:21.274701 (+ 704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":171,"threads_started":1}
I20260808 10:47:21.276892 21810 tablet_service.cc:1467] Processing CreateTablet for tablet 8f89b53d8cf04fc39c1587c1d022285d (DEFAULT_TABLE table=test-table [id=0849b0601c3347558dfceb5a74228595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:47:21.277930 21810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f89b53d8cf04fc39c1587c1d022285d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:21.277638 22196 tablet_service.cc:1467] Processing CreateTablet for tablet 8f89b53d8cf04fc39c1587c1d022285d (DEFAULT_TABLE table=test-table [id=0849b0601c3347558dfceb5a74228595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:47:21.277638 22010 tablet_service.cc:1467] Processing CreateTablet for tablet 8f89b53d8cf04fc39c1587c1d022285d (DEFAULT_TABLE table=test-table [id=0849b0601c3347558dfceb5a74228595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:47:21.278724 22010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f89b53d8cf04fc39c1587c1d022285d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:21.278730 22196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f89b53d8cf04fc39c1587c1d022285d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:21.284880 22317 tablet_bootstrap.cc:492] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Bootstrap starting.
I20260808 10:47:21.286813 22321 tablet_bootstrap.cc:492] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc: Bootstrap starting.
I20260808 10:47:21.286813 22319 tablet_bootstrap.cc:492] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f: Bootstrap starting.
I20260808 10:47:21.287364 22317 tablet_bootstrap.cc:654] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:21.288120 22317 log.cc:824] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:21.289541 22321 tablet_bootstrap.cc:654] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:21.289541 22319 tablet_bootstrap.cc:654] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:21.290234 22317 tablet_bootstrap.cc:492] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: No bootstrap required, opened a new log
I20260808 10:47:21.290464 22317 ts_tablet_manager.cc:1397] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:47:21.290433 22321 log.cc:824] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:21.290433 22319 log.cc:824] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:21.292270 22321 tablet_bootstrap.cc:492] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc: No bootstrap required, opened a new log
I20260808 10:47:21.292480 22321 ts_tablet_manager.cc:1397] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:47:21.292665 22319 tablet_bootstrap.cc:492] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f: No bootstrap required, opened a new log
I20260808 10:47:21.292877 22319 ts_tablet_manager.cc:1397] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:47:21.295253 22317 raft_consensus.cc:348] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } }
I20260808 10:47:21.295550 22317 raft_consensus.cc:374] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:21.295609 22317 raft_consensus.cc:729] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c53d5237a75840849a3ce6d7f10445d6, State: Initialized, Role: FOLLOWER
I20260808 10:47:21.295446 22321 raft_consensus.cc:348] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } }
I20260808 10:47:21.295712 22321 raft_consensus.cc:374] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:21.295778 22321 raft_consensus.cc:729] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d709d62000ff4579a6bf93923e440cdc, State: Initialized, Role: FOLLOWER
I20260808 10:47:21.296103 22317 consensus_queue.cc:260] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } }
I20260808 10:47:21.296272 22321 consensus_queue.cc:260] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } }
I20260808 10:47:21.297010 21913 heartbeater.cc:503] Master 127.31.237.190:34147 was elected leader, sending a full tablet report...
I20260808 10:47:21.296967 22319 raft_consensus.cc:348] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } }
I20260808 10:47:21.297201 22317 ts_tablet_manager.cc:1428] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 10:47:21.297210 22319 raft_consensus.cc:374] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:21.297258 22319 raft_consensus.cc:729] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59a4550bf97a4de6a4c1ea827be7e81f, State: Initialized, Role: FOLLOWER
I20260808 10:47:21.297277 22294 heartbeater.cc:503] Master 127.31.237.190:34147 was elected leader, sending a full tablet report...
I20260808 10:47:21.297654 22319 consensus_queue.cc:260] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } }
I20260808 10:47:21.298265 22106 heartbeater.cc:503] Master 127.31.237.190:34147 was elected leader, sending a full tablet report...
I20260808 10:47:21.298763 22319 ts_tablet_manager.cc:1428] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260808 10:47:21.298764 22321 ts_tablet_manager.cc:1428] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:47:21.445456 22326 raft_consensus.cc:482] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:47:21.445729 22326 raft_consensus.cc:504] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } }
I20260808 10:47:21.447129 22326 leader_election.cc:288] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977), d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973)
I20260808 10:47:21.451891 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.447429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:39973 (local address 127.31.237.129:40753)
0808 10:47:21.447881 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:21.447898 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:21.448065 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:21.448521 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:21.448525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:21.448544 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:21.448699 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:21.448706 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.449729 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.449733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:21.450602 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:21.450640 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.451441 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.451445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:21.451536 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:21.451566 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 10:47:21.451648 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:47:21.451717 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":3,"thread_start_us":135,"threads_started":1}
I20260808 10:47:21.451908 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.447510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:40753 (local address 127.31.237.131:39973)
0808 10:47:21.447652 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:21.447658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:21.448003 (+ 345us) server_negotiation.cc:407] Connection header received
0808 10:47:21.448251 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:21.448255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:21.448305 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:21.448357 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:21.448893 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.449609 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.450794 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.451301 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.451429 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:21.451482 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:21.451687 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 10:47:21.451771 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:47:21.452347 22332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.447429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:34977 (local address 127.31.237.129:36865)
0808 10:47:21.447993 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:21.448007 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:21.448139 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:21.448430 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:21.448433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:21.448449 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:21.448599 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:21.448605 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.449645 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.449648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:21.450956 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:21.450967 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.451892 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.451895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:21.452005 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:21.452028 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:47:21.452144 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 10:47:21.452210 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":241,"thread_start_us":109,"threads_started":1}
I20260808 10:47:21.452358 22313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.447657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:36865 (local address 127.31.237.130:34977)
0808 10:47:21.447784 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:21.447789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:21.448109 (+ 320us) server_negotiation.cc:407] Connection header received
0808 10:47:21.448251 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:21.448255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:21.448308 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:21.448409 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:21.448720 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.449530 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.451088 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.451665 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.451778 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:21.451821 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:21.452166 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 10:47:21.452241 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:47:21.453051 22223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f89b53d8cf04fc39c1587c1d022285d" candidate_uuid: "c53d5237a75840849a3ce6d7f10445d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d709d62000ff4579a6bf93923e440cdc" is_pre_election: true
I20260808 10:47:21.453136 22035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f89b53d8cf04fc39c1587c1d022285d" candidate_uuid: "c53d5237a75840849a3ce6d7f10445d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" is_pre_election: true
I20260808 10:47:21.453395 22035 raft_consensus.cc:2454] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c53d5237a75840849a3ce6d7f10445d6 in term 0.
I20260808 10:47:21.453395 22223 raft_consensus.cc:2454] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c53d5237a75840849a3ce6d7f10445d6 in term 0.
I20260808 10:47:21.453931 21752 leader_election.cc:302] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 59a4550bf97a4de6a4c1ea827be7e81f, c53d5237a75840849a3ce6d7f10445d6, d709d62000ff4579a6bf93923e440cdc; no voters:
I20260808 10:47:21.454239 22326 raft_consensus.cc:2781] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:47:21.454324 22326 raft_consensus.cc:482] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:47:21.454367 22326 raft_consensus.cc:3037] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:21.455479 22326 raft_consensus.cc:504] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } }
I20260808 10:47:21.455825 22326 leader_election.cc:288] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 1 election: Requested vote from peers 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977), d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973)
I20260808 10:47:21.456112 22035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f89b53d8cf04fc39c1587c1d022285d" candidate_uuid: "c53d5237a75840849a3ce6d7f10445d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59a4550bf97a4de6a4c1ea827be7e81f"
I20260808 10:47:21.456199 22223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f89b53d8cf04fc39c1587c1d022285d" candidate_uuid: "c53d5237a75840849a3ce6d7f10445d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d709d62000ff4579a6bf93923e440cdc"
I20260808 10:47:21.456264 22035 raft_consensus.cc:3037] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:21.456327 22223 raft_consensus.cc:3037] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:21.457365 22035 raft_consensus.cc:2454] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c53d5237a75840849a3ce6d7f10445d6 in term 1.
I20260808 10:47:21.457595 22223 raft_consensus.cc:2454] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c53d5237a75840849a3ce6d7f10445d6 in term 1.
I20260808 10:47:21.457690 21749 leader_election.cc:302] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59a4550bf97a4de6a4c1ea827be7e81f, c53d5237a75840849a3ce6d7f10445d6; no voters:
I20260808 10:47:21.457887 22326 raft_consensus.cc:2781] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:47:21.458140 22326 raft_consensus.cc:686] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 1 LEADER]: Becoming Leader. State: Replica: c53d5237a75840849a3ce6d7f10445d6, State: Running, Role: LEADER
I20260808 10:47:21.458482 22326 consensus_queue.cc:237] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } }
I20260808 10:47:21.461906 21628 catalog_manager.cc:5495] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c53d5237a75840849a3ce6d7f10445d6 (127.31.237.129). New cstate: current_term: 1 leader_uuid: "c53d5237a75840849a3ce6d7f10445d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } health_report { overall_health: UNKNOWN } } }
W20260808 10:47:21.468708 22295 tablet.cc:2369] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:47:21.478960 22108 tablet.cc:2369] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:47:21.495921 21914 tablet.cc:2369] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:47:21.508818 22312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.504877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.31.237.129:37879)
0808 10:47:21.505041 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:21.505046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:21.505060 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:47:21.505276 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:21.505279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:21.505358 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:21.505476 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:47:21.505743 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.506267 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.507232 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:21.507469 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:21.507583 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:21.507936 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:47:21.508405 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:47:21.508554 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:21.508597 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 10:47:21.508661 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:47:21.515264 22035 raft_consensus.cc:1261] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f [term 1 FOLLOWER]: Refusing update from remote peer c53d5237a75840849a3ce6d7f10445d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:47:21.515264 22223 raft_consensus.cc:1261] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc [term 1 FOLLOWER]: Refusing update from remote peer c53d5237a75840849a3ce6d7f10445d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:47:21.516109 22333 consensus_queue.cc:1035] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:47:21.516290 22326 consensus_queue.cc:1035] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:47:21.524148 22337 mvcc.cc:205] Tried to move back new op lower bound from 7316218026038472704 to 7316218025816645632. Current Snapshot: MvccSnapshot[applied={T|T < 7316218026038472704}]
I20260808 10:47:21.524865 22341 mvcc.cc:205] Tried to move back new op lower bound from 7316218026038472704 to 7316218025816645632. Current Snapshot: MvccSnapshot[applied={T|T < 7316218026038472704}]
I20260808 10:47:21.527931 22340 mvcc.cc:205] Tried to move back new op lower bound from 7316218026038472704 to 7316218025816645632. Current Snapshot: MvccSnapshot[applied={T|T < 7316218026038472704}]
I20260808 10:47:21.534996 21628 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:47:21.535176 21628 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:47:21.537744 21628 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Sending DeleteTablet for 3 replicas of tablet 8f89b53d8cf04fc39c1587c1d022285d
I20260808 10:47:21.538558 22010 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f89b53d8cf04fc39c1587c1d022285d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:47:21 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:54816
I20260808 10:47:21.538779 22196 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f89b53d8cf04fc39c1587c1d022285d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:47:21 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:50264
I20260808 10:47:21.538780 21810 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f89b53d8cf04fc39c1587c1d022285d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:47:21 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:43076
I20260808 10:47:21.539218 22354 tablet_replica.cc:330] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc: stopping tablet replica
I20260808 10:47:21.539449 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 21720
I20260808 10:47:21.539453 22355 tablet_replica.cc:330] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: stopping tablet replica
I20260808 10:47:21.539575 22354 raft_consensus.cc:2229] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:47:21.539899 22354 raft_consensus.cc:2258] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:47:21.539630 22353 tablet_replica.cc:330] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f: stopping tablet replica
I20260808 10:47:21.546325 22353 raft_consensus.cc:2229] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:47:21.546866 22353 raft_consensus.cc:2258] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:47:21.548305 22353 ts_tablet_manager.cc:1905] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:47:21.548365 22354 ts_tablet_manager.cc:1905] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:47:21.550832 22353 ts_tablet_manager.cc:1918] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:47:21.550971 22353 log.cc:1196] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/8f89b53d8cf04fc39c1587c1d022285d
I20260808 10:47:21.551316 22353 ts_tablet_manager.cc:1939] T 8f89b53d8cf04fc39c1587c1d022285d P 59a4550bf97a4de6a4c1ea827be7e81f: Deleting consensus metadata
I20260808 10:47:21.552197 22354 ts_tablet_manager.cc:1918] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:47:21.552326 22354 log.cc:1196] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/8f89b53d8cf04fc39c1587c1d022285d
I20260808 10:47:21.552527 21606 catalog_manager.cc:4841] TS 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977): tablet 8f89b53d8cf04fc39c1587c1d022285d (table test-table [id=0849b0601c3347558dfceb5a74228595]) successfully deleted
I20260808 10:47:21.552696 22354 ts_tablet_manager.cc:1939] T 8f89b53d8cf04fc39c1587c1d022285d P d709d62000ff4579a6bf93923e440cdc: Deleting consensus metadata
I20260808 10:47:21.554163 21607 catalog_manager.cc:4841] TS d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973): tablet 8f89b53d8cf04fc39c1587c1d022285d (table test-table [id=0849b0601c3347558dfceb5a74228595]) successfully deleted
W20260808 10:47:21.555269 21605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:37879 (error 108)
I20260808 10:47:21.555531 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 21919
I20260808 10:47:21.561324 22315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:21.560951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:37879 (local address 127.0.0.1:43108)
0808 10:47:21.561124 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:21.561214 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:37879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260808 10:47:21.561537 21605 catalog_manager.cc:4584] TS c53d5237a75840849a3ce6d7f10445d6 (127.31.237.129:37879): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8f89b53d8cf04fc39c1587c1d022285d: Network error: Client connection negotiation failed: client connection to 127.31.237.129:37879: connect: Connection refused (error 111)
I20260808 10:47:21.569595 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 22112
I20260808 10:47:21.583410 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 21583
I20260808 10:47:21.603128 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:34147
--webserver_interface=127.31.237.190
--webserver_port=35951
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:34147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:21.724555 22358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:21.724848 22358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:21.724890 22358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:21.729034 22358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:47:21.729101 22358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:21.729120 22358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:21.729138 22358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:47:21.729153 22358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:47:21.734534 22358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:34147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:34147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=35951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:21.735694 22358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:21.736763 22358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:21.752475 22367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:21.752588 22358 server_base.cc:1034] running on GCE node
W20260808 10:47:21.752475 22373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:21.752514 22368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:21.753533 22358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:21.754781 22358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:21.755991 22358 hybrid_clock.cc:648] HybridClock initialized: now 1786186041755969 us; error 35 us; skew 500 ppm
Aug 08 10:47:21 dist-test-slave-twwt krb5kdc[21564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186041, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:21.759253 22358 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:47:21.760351 22358 webserver.cc:460] Webserver started at http://127.31.237.190:35951/ using document root <none> and password file <none>
I20260808 10:47:21.761019 22358 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:21.761147 22358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:21.765604 22358 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:47:21.768332 22382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:21.770291 22358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:47:21.770479 22358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "0e53e08be91e402aba993003729aca96"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "4e50087ef5cae52dfb4378f3e11d7456"
server_key_iv: "e34f8b61c7ffa7ff8d9a30f0d005218a"
server_key_version: "encryptionkey@0"
I20260808 10:47:21.771519 22358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:21.822494 22358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:21.827195 22358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:21.827453 22358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:21.835670 22358 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:34147
I20260808 10:47:21.835964 22452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:34147 every 8 connection(s)
I20260808 10:47:21.836928 22358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:47:21.841562 22453 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:47:21.843495 22453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Bootstrap starting.
I20260808 10:47:21.844660 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 22358
I20260808 10:47:21.845229 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:37879
--local_ip_for_outbound_sockets=127.31.237.129
--tserver_master_addrs=127.31.237.190:34147
--webserver_port=43729
--webserver_interface=127.31.237.129
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:47:21.862274 22453 log.cc:824] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:21.871793 22453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:47:21.872224 22453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Bootstrap complete.
I20260808 10:47:21.876677 22453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } }
I20260808 10:47:21.877240 22453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e53e08be91e402aba993003729aca96, State: Initialized, Role: FOLLOWER
I20260808 10:47:21.877892 22453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } }
I20260808 10:47:21.878105 22453 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:47:21.878216 22453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:47:21.878358 22453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:47:21.880903 22453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } }
I20260808 10:47:21.881273 22453 leader_election.cc:302] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e53e08be91e402aba993003729aca96; no voters:
I20260808 10:47:21.881677 22453 leader_election.cc:288] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:47:21.881774 22460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:47:21.881995 22460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 2 LEADER]: Becoming Leader. State: Replica: 0e53e08be91e402aba993003729aca96, State: Running, Role: LEADER
I20260808 10:47:21.882300 22460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } }
I20260808 10:47:21.883082 22453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:47:21.883363 22460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e53e08be91e402aba993003729aca96. Latest consensus state: current_term: 2 leader_uuid: "0e53e08be91e402aba993003729aca96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } } }
I20260808 10:47:21.883481 22460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:21.883899 22462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0e53e08be91e402aba993003729aca96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } } }
I20260808 10:47:21.884042 22462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:21.888590 22480 catalog_manager.cc:1259] Loaded cluster ID: 12232d378e5541e395c50311b32274b8
I20260808 10:47:21.888653 22480 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: loading cluster ID for follower catalog manager: success
W20260808 10:47:21.889564 22480 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260808 10:47:21.889629 22480 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260808 10:47:21.889693 22465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:47:21.891922 22465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:47:21.892375 22465 catalog_manager.cc:1259] Loaded cluster ID: 12232d378e5541e395c50311b32274b8
I20260808 10:47:21.892441 22465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:47:21.893323 22465 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 10:47:21.896102 22465 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 10:47:21.901620 22465 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:47:21.902338 22465 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Loaded TSK: 0
I20260808 10:47:21.903429 22465 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:47:21.980594 22456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:21.980988 22456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:21.981112 22456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:21.986137 22456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:21.986246 22456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:21.986349 22456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:47:21.992194 22456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:37879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=43729
--tserver_master_addrs=127.31.237.190:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:21.993431 22456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:21.994551 22456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:22.004998 22489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:22.005016 22488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:22.005364 22456 server_base.cc:1034] running on GCE node
W20260808 10:47:22.004998 22493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:22.005885 22456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:22.006788 22456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:22.007999 22456 hybrid_clock.cc:648] HybridClock initialized: now 1786186042007957 us; error 51 us; skew 500 ppm
Aug 08 10:47:22 dist-test-slave-twwt krb5kdc[21564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186042, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:22.011679 22456 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:47:22.012794 22456 webserver.cc:460] Webserver started at http://127.31.237.129:43729/ using document root <none> and password file <none>
I20260808 10:47:22.013451 22456 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:22.013535 22456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:22.017798 22456 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260808 10:47:22.020220 22500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:22.021618 22456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:47:22.021723 22456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "c53d5237a75840849a3ce6d7f10445d6"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "3c8e45c0e5552c52e3f1eca8da8242db"
server_key_iv: "2124f5265c2f0d1b19e5ef08c35017d8"
server_key_version: "encryptionkey@0"
I20260808 10:47:22.022325 22456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:22.042505 22456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:22.046399 22456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:22.046658 22456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:22.047566 22456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:22.049291 22510 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:47:22.055428 22456 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:47:22.055500 22456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20260808 10:47:22.055590 22456 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:47:22.058434 22456 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:47:22.058498 22456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260808 10:47:22.058885 22510 tablet_bootstrap.cc:492] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Bootstrap starting.
I20260808 10:47:22.069542 22456 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:37879
I20260808 10:47:22.069880 22668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:37879 every 8 connection(s)
I20260808 10:47:22.070689 22456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:47:22.074066 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 22456
I20260808 10:47:22.074512 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:34977
--local_ip_for_outbound_sockets=127.31.237.130
--tserver_master_addrs=127.31.237.190:34147
--webserver_port=38361
--webserver_interface=127.31.237.130
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:47:22.081602 22510 log.cc:824] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:47:22 dist-test-slave-twwt krb5kdc[21564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186042, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:22.091650 22672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.073157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:57727 (local address 127.31.237.190:34147)
0808 10:47:22.073702 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:22.073722 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:22.073771 (+ 49us) server_negotiation.cc:407] Connection header received
0808 10:47:22.076399 (+ 2628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:22.076416 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:22.076851 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:22.077302 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:22.078869 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.080488 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.082469 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.083078 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.083277 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.086253 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:22.086265 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:22.086279 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:22.086311 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:22.088186 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:22.088775 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:22.088781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:22.088790 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:22.088924 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:22.089300 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:22.089304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:22.089306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:22.089685 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:22.089845 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:22.090420 (+ 575us) server_negotiation.cc:299] Negotiation successful
0808 10:47:22.090785 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":153,"threads_started":1}
I20260808 10:47:22.091662 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.072977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:57727)
0808 10:47:22.073583 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:22.073637 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:22.075834 (+ 2197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:22.077392 (+ 1558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:22.077404 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:22.078006 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:22.078656 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.078679 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.080677 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.080684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.082289 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.082306 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.083242 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.083246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.083442 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.084216 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:22.084242 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:22.086033 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:22.088344 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:22.088350 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:22.088364 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:22.088641 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:22.089036 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:22.089040 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:22.089042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:22.089195 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:22.089816 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:22.089823 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:22.090087 (+ 264us) client_negotiation.cc:769] Sending connection context
0808 10:47:22.090296 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 10:47:22.090692 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":255,"threads_started":1}
I20260808 10:47:22.091842 22510 tablet_bootstrap.cc:492] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:47:22.092657 22510 tablet_bootstrap.cc:492] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Bootstrap complete.
I20260808 10:47:22.094341 22510 ts_tablet_manager.cc:1397] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Time spent bootstrapping tablet: real 0.036s user 0.025s sys 0.007s
I20260808 10:47:22.094502 22669 heartbeater.cc:347] Connected to a master server at 127.31.237.190:34147
I20260808 10:47:22.094797 22669 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:22.095317 22669 heartbeater.cc:511] Master 127.31.237.190:34147 requested a full tablet report, sending...
I20260808 10:47:22.097765 22406 ts_manager.cc:194] Registered new tserver with Master: c53d5237a75840849a3ce6d7f10445d6 (127.31.237.129:37879)
I20260808 10:47:22.097748 22510 raft_consensus.cc:348] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } }
I20260808 10:47:22.098354 22510 raft_consensus.cc:729] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c53d5237a75840849a3ce6d7f10445d6, State: Initialized, Role: FOLLOWER
I20260808 10:47:22.098892 22510 consensus_queue.cc:260] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } }
I20260808 10:47:22.099954 22510 ts_tablet_manager.cc:1428] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:47:22.101969 22406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:57727
I20260808 10:47:22.104676 22669 heartbeater.cc:503] Master 127.31.237.190:34147 was elected leader, sending a full tablet report...
I20260808 10:47:22.108021 22678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.101135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43110 (local address 127.31.237.129:37879)
0808 10:47:22.101428 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:22.101440 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:22.103091 (+ 1651us) server_negotiation.cc:407] Connection header received
0808 10:47:22.103293 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:22.103299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:22.103789 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:22.103955 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:22.104308 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.105284 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.106918 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.107411 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.107521 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.107611 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:22.107844 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 10:47:22.107916 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":101,"threads_started":1}
I20260808 10:47:22.108070 22677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.101041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:37879 (local address 127.0.0.1:43110)
0808 10:47:22.102934 (+ 1893us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:22.102983 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:22.103131 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:22.103982 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:22.103988 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:22.104018 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:22.104156 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.104167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.105415 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.105419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.106795 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.106808 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.107540 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.107543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.107655 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.107696 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 10:47:22.107808 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 10:47:22.107932 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1697,"thread_start_us":148,"threads_started":1}
I20260808 10:47:22.109252 22576 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f89b53d8cf04fc39c1587c1d022285d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:47:21 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:43110
I20260808 10:47:22.110220 22680 tablet_replica.cc:330] stopping tablet replica
I20260808 10:47:22.110522 22680 raft_consensus.cc:2229] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:47:22.110802 22680 raft_consensus.cc:2258] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:47:22.111796 22680 ts_tablet_manager.cc:1905] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:47:22.115471 22680 ts_tablet_manager.cc:1918] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:47:22.115590 22680 log.cc:1196] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal/wals/8f89b53d8cf04fc39c1587c1d022285d
I20260808 10:47:22.115898 22680 ts_tablet_manager.cc:1939] T 8f89b53d8cf04fc39c1587c1d022285d P c53d5237a75840849a3ce6d7f10445d6: Deleting consensus metadata
I20260808 10:47:22.117574 22386 catalog_manager.cc:4841] TS c53d5237a75840849a3ce6d7f10445d6 (127.31.237.129:37879): tablet 8f89b53d8cf04fc39c1587c1d022285d (table test-table [id=0849b0601c3347558dfceb5a74228595]) successfully deleted
W20260808 10:47:22.205528 22673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:22.205816 22673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:22.205869 22673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:22.210196 22673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:22.210260 22673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:22.210321 22673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:47:22.215909 22673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:34977
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=38361
--tserver_master_addrs=127.31.237.190:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:22.217072 22673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:22.218151 22673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:22.228767 22687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:22.228788 22689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:22.228767 22686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:22.229202 22673 server_base.cc:1034] running on GCE node
I20260808 10:47:22.229615 22673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:22.230296 22673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:22.231503 22673 hybrid_clock.cc:648] HybridClock initialized: now 1786186042231459 us; error 63 us; skew 500 ppm
Aug 08 10:47:22 dist-test-slave-twwt krb5kdc[21564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186042, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:22.234721 22673 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:47:22.235672 22673 webserver.cc:460] Webserver started at http://127.31.237.130:38361/ using document root <none> and password file <none>
I20260808 10:47:22.236310 22673 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:22.236434 22673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:22.240494 22673 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:47:22.242970 22697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:22.244379 22673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:47:22.244531 22673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "59a4550bf97a4de6a4c1ea827be7e81f"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "7a2a147f820e869dc60dd50ba8006685"
server_key_iv: "5b6b784ee0c5a90ca50766dd36d4ad24"
server_key_version: "encryptionkey@0"
I20260808 10:47:22.245131 22673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:22.256758 22673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:22.260304 22673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:22.260553 22673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:22.261240 22673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:22.262202 22673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:22.262295 22673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:22.262367 22673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:22.262425 22673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:22.272624 22673 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:34977
I20260808 10:47:22.272655 22863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:34977 every 8 connection(s)
I20260808 10:47:22.273694 22673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:47:22.274461 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 22673
I20260808 10:47:22.274996 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:39973
--local_ip_for_outbound_sockets=127.31.237.131
--tserver_master_addrs=127.31.237.190:34147
--webserver_port=46323
--webserver_interface=127.31.237.131
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:47:22 dist-test-slave-twwt krb5kdc[21564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186042, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:22.290897 22672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.275708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:56269 (local address 127.31.237.190:34147)
0808 10:47:22.275861 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:22.275866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:22.277088 (+ 1222us) server_negotiation.cc:407] Connection header received
0808 10:47:22.279242 (+ 2154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:22.279249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:22.279319 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:22.279422 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:22.280865 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.281796 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.283095 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.283596 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.283730 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.286820 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:22.286826 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:22.286831 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:22.286877 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:22.288655 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:22.289249 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:22.289253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:22.289255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:22.289348 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:22.289684 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:22.289689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:22.289694 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:22.289933 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:22.290029 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:22.290547 (+ 518us) server_negotiation.cc:299] Negotiation successful
0808 10:47:22.290709 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:47:22.291898 22866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.276273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:56269)
0808 10:47:22.276888 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:22.276958 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:22.278994 (+ 2036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:22.279607 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:22.279614 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:22.279991 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:22.280671 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.280692 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.281950 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.281953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.282959 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.282973 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.283752 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.283755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.283942 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.284668 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:22.284696 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:22.286576 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:22.288798 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:22.288804 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:22.288817 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:22.289123 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:22.289442 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:22.289444 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:22.289446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:22.289583 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:22.290037 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:22.290042 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:22.290296 (+ 254us) client_negotiation.cc:769] Sending connection context
0808 10:47:22.290462 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 10:47:22.290918 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":159,"threads_started":1}
I20260808 10:47:22.293203 22864 heartbeater.cc:347] Connected to a master server at 127.31.237.190:34147
I20260808 10:47:22.293458 22864 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:22.293996 22864 heartbeater.cc:511] Master 127.31.237.190:34147 requested a full tablet report, sending...
I20260808 10:47:22.295193 22406 ts_manager.cc:194] Registered new tserver with Master: 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977)
I20260808 10:47:22.295819 22406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:56269
W20260808 10:47:22.399935 22868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:22.400223 22868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:22.400266 22868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:22.404567 22868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:22.404629 22868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:22.404722 22868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:47:22.410331 22868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:39973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=46323
--tserver_master_addrs=127.31.237.190:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:22.411530 22868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:22.412621 22868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:22.423293 22882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:22.423290 22880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:22.423357 22879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:22.423784 22868 server_base.cc:1034] running on GCE node
I20260808 10:47:22.424190 22868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:22.424974 22868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:22.426220 22868 hybrid_clock.cc:648] HybridClock initialized: now 1786186042426188 us; error 32 us; skew 500 ppm
Aug 08 10:47:22 dist-test-slave-twwt krb5kdc[21564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186042, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:22.429116 22868 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:47:22.429977 22868 webserver.cc:460] Webserver started at http://127.31.237.131:46323/ using document root <none> and password file <none>
I20260808 10:47:22.430552 22868 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:22.430693 22868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:22.434564 22868 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:47:22.436793 22891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:22.438020 22868 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:47:22.438160 22868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "d709d62000ff4579a6bf93923e440cdc"
format_stamp: "Formatted at 2026-08-08 10:47:21 on dist-test-slave-twwt"
server_key: "ad1142777267a1b3b6eb7cba12810e0e"
server_key_iv: "2924571fcf10922815afeb0f24f395ba"
server_key_version: "encryptionkey@0"
I20260808 10:47:22.438777 22868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:22.462365 22868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:22.466109 22868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:22.466341 22868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:22.467032 22868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:22.468359 22868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:22.468436 22868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:22.468508 22868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:22.468538 22868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:22.477931 22868 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:39973
I20260808 10:47:22.477958 23074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:39973 every 8 connection(s)
I20260808 10:47:22.479045 22868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:47:22.484246 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 22868
Aug 08 10:47:22 dist-test-slave-twwt krb5kdc[21564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186042, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:22.494102 23082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.484992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53520 (local address 127.31.237.190:34147)
0808 10:47:22.485641 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:22.485646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:22.485657 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:47:22.485689 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:22.485692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:22.485729 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:22.485814 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:22.486755 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.487289 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.488473 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.488717 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.488831 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.490471 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:22.490475 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:22.490477 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:22.490503 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:22.492323 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:22.492800 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:22.492805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:22.492807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:22.492906 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:22.493200 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:22.493203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:22.493205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:22.493388 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:22.493639 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:22.493782 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 10:47:22.493915 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":122,"threads_started":1}
I20260808 10:47:22.494614 22672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.480966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:52009 (local address 127.31.237.190:34147)
0808 10:47:22.481155 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:22.481160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:22.482146 (+ 986us) server_negotiation.cc:407] Connection header received
0808 10:47:22.483481 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:22.483485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:22.483581 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:22.483666 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:22.485225 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.485999 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.487422 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.487648 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.487737 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.490816 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:22.490819 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:22.490821 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:22.490881 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:22.492447 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:22.492994 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:22.492998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:22.493001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:22.493080 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:22.493477 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:22.493480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:22.493482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:22.493649 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:22.493758 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:22.494345 (+ 587us) server_negotiation.cc:299] Negotiation successful
0808 10:47:22.494452 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 10:47:22.495935 23077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.481401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:52009)
0808 10:47:22.481887 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:22.481930 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:22.483245 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:22.483837 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:22.483844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:22.484274 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:22.485019 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.485046 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.486136 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.486141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.487282 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.487292 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.487805 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.487809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.488014 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.488732 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:22.488749 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:22.490589 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:22.492574 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:22.492580 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:22.492595 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:22.492879 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:22.493186 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:22.493189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:22.493191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:22.493335 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:22.493775 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:22.493782 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:22.494072 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 10:47:22.494289 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 10:47:22.495100 (+ 811us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":121,"threads_started":1}
I20260808 10:47:22.497041 23075 heartbeater.cc:347] Connected to a master server at 127.31.237.190:34147
I20260808 10:47:22.497305 23075 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:22.497797 23075 heartbeater.cc:511] Master 127.31.237.190:34147 requested a full tablet report, sending...
I20260808 10:47:22.498842 22406 ts_manager.cc:194] Registered new tserver with Master: d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973)
I20260808 10:47:22.499384 22406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:52009
I20260808 10:47:22.507870 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:47:22.512140 22672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.508474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53534 (local address 127.31.237.190:34147)
0808 10:47:22.508619 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:22.508622 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:22.508803 (+ 181us) server_negotiation.cc:407] Connection header received
0808 10:47:22.508947 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:22.508950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:22.509046 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:22.509135 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:47:22.509368 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.509855 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.510737 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.510986 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.511079 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.511368 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:47:22.511777 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:47:22.511885 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:22.511973 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 10:47:22.512030 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:47:22.512874 22406 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:53534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:47:22.516295 22406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:47:22.525130 22576 tablet_service.cc:1467] Processing CreateTablet for tablet 246e5c2d8644461cb3b71b2acdf85bd7 (DEFAULT_TABLE table=test-table [id=88ad4249a80240b683f38079093e8ad7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:47:22.525924 22576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 246e5c2d8644461cb3b71b2acdf85bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:22.529109 22510 tablet_bootstrap.cc:492] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6: Bootstrap starting.
I20260808 10:47:22.529474 22677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.523771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:39973 (local address 127.0.0.1:44042)
0808 10:47:22.523944 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:22.523960 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:22.524045 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:22.524959 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:22.524963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:22.524983 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:22.525091 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.525098 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.526675 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.526679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.527775 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.527789 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.529098 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.529102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.529195 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.529223 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 10:47:22.529277 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:47:22.529351 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 10:47:22.530088 22510 tablet_bootstrap.cc:654] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:22.530241 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.524016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44042 (local address 127.31.237.131:39973)
0808 10:47:22.524390 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:22.524410 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:22.524440 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:47:22.524530 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:22.524540 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:22.524786 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:22.524961 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:22.525203 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.526516 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.527917 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.528963 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.529768 (+ 805us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.529921 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:22.529986 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 10:47:22.530097 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":137,"threads_started":1}
I20260808 10:47:22.530412 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.524093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:34977 (local address 127.0.0.1:41002)
0808 10:47:22.524411 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:22.524425 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:22.524511 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:22.526067 (+ 1556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:22.526070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:22.526088 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:22.526173 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.526178 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.527677 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.527684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.528816 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.528827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.530038 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.530041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.530126 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.530149 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:47:22.530217 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:47:22.530283 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":111,"threads_started":1}
I20260808 10:47:22.530470 23092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.524470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41002 (local address 127.31.237.130:34977)
0808 10:47:22.524903 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:22.524916 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:22.524938 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:47:22.525014 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:22.525019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:22.525389 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:22.526308 (+ 919us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:22.526457 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.527533 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.528962 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.529897 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.530052 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.530185 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:22.530240 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 10:47:22.530323 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":129,"threads_started":1}
I20260808 10:47:22.531314 22510 tablet_bootstrap.cc:492] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6: No bootstrap required, opened a new log
I20260808 10:47:22.531423 22510 ts_tablet_manager.cc:1397] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:47:22.531810 22510 raft_consensus.cc:348] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:22.531922 22510 raft_consensus.cc:374] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:22.531961 22510 raft_consensus.cc:729] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c53d5237a75840849a3ce6d7f10445d6, State: Initialized, Role: FOLLOWER
I20260808 10:47:22.532092 22510 consensus_queue.cc:260] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:22.532358 22510 ts_tablet_manager.cc:1428] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 10:47:22.533336 22976 tablet_service.cc:1467] Processing CreateTablet for tablet 246e5c2d8644461cb3b71b2acdf85bd7 (DEFAULT_TABLE table=test-table [id=88ad4249a80240b683f38079093e8ad7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:47:22.533514 22770 tablet_service.cc:1467] Processing CreateTablet for tablet 246e5c2d8644461cb3b71b2acdf85bd7 (DEFAULT_TABLE table=test-table [id=88ad4249a80240b683f38079093e8ad7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:47:22.534386 22976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 246e5c2d8644461cb3b71b2acdf85bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:22.534538 22770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 246e5c2d8644461cb3b71b2acdf85bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:22.541436 23095 tablet_bootstrap.cc:492] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f: Bootstrap starting.
I20260808 10:47:22.541436 23096 tablet_bootstrap.cc:492] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc: Bootstrap starting.
I20260808 10:47:22.544278 23096 tablet_bootstrap.cc:654] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:22.544322 23095 tablet_bootstrap.cc:654] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:22.545042 23096 log.cc:824] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:22.545042 23095 log.cc:824] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:22.546977 23095 tablet_bootstrap.cc:492] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f: No bootstrap required, opened a new log
I20260808 10:47:22.546986 23096 tablet_bootstrap.cc:492] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc: No bootstrap required, opened a new log
I20260808 10:47:22.547181 23096 ts_tablet_manager.cc:1397] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:47:22.547185 23095 ts_tablet_manager.cc:1397] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:47:22.550462 23096 raft_consensus.cc:348] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:22.550521 23095 raft_consensus.cc:348] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:22.550770 23096 raft_consensus.cc:374] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:22.550796 23095 raft_consensus.cc:374] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:22.550850 23095 raft_consensus.cc:729] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59a4550bf97a4de6a4c1ea827be7e81f, State: Initialized, Role: FOLLOWER
I20260808 10:47:22.550854 23096 raft_consensus.cc:729] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d709d62000ff4579a6bf93923e440cdc, State: Initialized, Role: FOLLOWER
I20260808 10:47:22.551247 23096 consensus_queue.cc:260] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:22.551282 23095 consensus_queue.cc:260] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:22.551926 23075 heartbeater.cc:503] Master 127.31.237.190:34147 was elected leader, sending a full tablet report...
I20260808 10:47:22.551926 22864 heartbeater.cc:503] Master 127.31.237.190:34147 was elected leader, sending a full tablet report...
I20260808 10:47:22.552315 23095 ts_tablet_manager.cc:1428] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 10:47:22.553390 23096 ts_tablet_manager.cc:1428] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260808 10:47:22.572289 22670 tablet.cc:2369] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:47:22.679652 23103 raft_consensus.cc:482] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:47:22.679899 23103 raft_consensus.cc:504] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:22.681313 23103 leader_election.cc:288] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977), c53d5237a75840849a3ce6d7f10445d6 (127.31.237.129:37879)
I20260808 10:47:22.685650 23077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.681510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:34977 (local address 127.31.237.131:33307)
0808 10:47:22.681753 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:22.681772 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:22.681875 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:22.682203 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:22.682206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:22.682229 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:22.682366 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.682377 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.683298 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.683302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.684366 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.684380 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.685200 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.685204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.685312 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.685339 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:47:22.685417 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 10:47:22.685507 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260808 10:47:22.685665 23092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.681666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:33307 (local address 127.31.237.130:34977)
0808 10:47:22.681793 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:22.681798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:22.681817 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:47:22.682010 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:22.682014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:22.682074 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:22.682167 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:22.682507 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.683132 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.684495 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.685026 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.685194 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.685256 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:22.685455 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 10:47:22.685536 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:47:22.685679 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.681627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:37879 (local address 127.31.237.131:47627)
0808 10:47:22.682065 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:22.682080 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:22.682181 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:22.682486 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:22.682489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:22.682508 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:22.682619 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.682646 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.683540 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.683543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.684549 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:22.684560 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.685248 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.685252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:22.685366 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.685407 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 10:47:22.685490 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:47:22.685555 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":131,"threads_started":1}
I20260808 10:47:22.685742 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.681755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:47627 (local address 127.31.237.129:37879)
0808 10:47:22.682064 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:22.682068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:22.682175 (+ 107us) server_negotiation.cc:407] Connection header received
0808 10:47:22.682268 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:22.682271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:22.682324 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:22.682431 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:22.682764 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.683428 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.684671 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.685097 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.685243 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.685298 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:22.685530 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 10:47:22.685606 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":117,"threads_started":1}
I20260808 10:47:22.686329 22609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246e5c2d8644461cb3b71b2acdf85bd7" candidate_uuid: "d709d62000ff4579a6bf93923e440cdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c53d5237a75840849a3ce6d7f10445d6" is_pre_election: true
I20260808 10:47:22.686738 22609 raft_consensus.cc:2454] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d709d62000ff4579a6bf93923e440cdc in term 0.
I20260808 10:47:22.686731 22798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246e5c2d8644461cb3b71b2acdf85bd7" candidate_uuid: "d709d62000ff4579a6bf93923e440cdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" is_pre_election: true
I20260808 10:47:22.687019 22798 raft_consensus.cc:2454] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d709d62000ff4579a6bf93923e440cdc in term 0.
I20260808 10:47:22.687268 22895 leader_election.cc:302] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c53d5237a75840849a3ce6d7f10445d6, d709d62000ff4579a6bf93923e440cdc; no voters:
I20260808 10:47:22.687572 23103 raft_consensus.cc:2781] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:47:22.687656 23103 raft_consensus.cc:482] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:47:22.687688 23103 raft_consensus.cc:3037] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:22.688854 23103 raft_consensus.cc:504] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:22.689230 23103 leader_election.cc:288] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [CANDIDATE]: Term 1 election: Requested vote from peers 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977), c53d5237a75840849a3ce6d7f10445d6 (127.31.237.129:37879)
I20260808 10:47:22.689623 22798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246e5c2d8644461cb3b71b2acdf85bd7" candidate_uuid: "d709d62000ff4579a6bf93923e440cdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59a4550bf97a4de6a4c1ea827be7e81f"
I20260808 10:47:22.689781 22798 raft_consensus.cc:3037] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:22.689824 22609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246e5c2d8644461cb3b71b2acdf85bd7" candidate_uuid: "d709d62000ff4579a6bf93923e440cdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c53d5237a75840849a3ce6d7f10445d6"
I20260808 10:47:22.689949 22609 raft_consensus.cc:3037] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:22.691284 22798 raft_consensus.cc:2454] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d709d62000ff4579a6bf93923e440cdc in term 1.
I20260808 10:47:22.691284 22609 raft_consensus.cc:2454] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d709d62000ff4579a6bf93923e440cdc in term 1.
I20260808 10:47:22.691573 22896 leader_election.cc:302] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59a4550bf97a4de6a4c1ea827be7e81f, d709d62000ff4579a6bf93923e440cdc; no voters:
I20260808 10:47:22.691817 23103 raft_consensus.cc:2781] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:47:22.692018 23103 raft_consensus.cc:686] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [term 1 LEADER]: Becoming Leader. State: Replica: d709d62000ff4579a6bf93923e440cdc, State: Running, Role: LEADER
I20260808 10:47:22.692387 23103 consensus_queue.cc:237] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:22.695888 22404 catalog_manager.cc:5495] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc reported cstate change: term changed from 0 to 1, leader changed from <none> to d709d62000ff4579a6bf93923e440cdc (127.31.237.131). New cstate: current_term: 1 leader_uuid: "d709d62000ff4579a6bf93923e440cdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } health_report { overall_health: UNKNOWN } } }
I20260808 10:47:22.725548 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.722124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44054 (local address 127.31.237.131:39973)
0808 10:47:22.722263 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:22.722274 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:22.722291 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:47:22.722373 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:22.722376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:22.722431 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:22.722526 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:47:22.722805 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.723365 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.724186 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:22.724405 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:22.724537 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:22.724751 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:47:22.725184 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:47:22.725278 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:22.725339 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 10:47:22.725406 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 10:47:22.729995 23076 tablet.cc:2369] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:47:22.732074 22609 raft_consensus.cc:1261] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Refusing update from remote peer d709d62000ff4579a6bf93923e440cdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:47:22.732079 22798 raft_consensus.cc:1261] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f [term 1 FOLLOWER]: Refusing update from remote peer d709d62000ff4579a6bf93923e440cdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:47:22.732846 23111 consensus_queue.cc:1035] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:47:22.733067 23103 consensus_queue.cc:1035] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:47:22.748234 22404 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:53534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:47:22.748399 22404 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:53534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:47:22.750928 22404 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Sending DeleteTablet for 3 replicas of tablet 246e5c2d8644461cb3b71b2acdf85bd7
I20260808 10:47:22.751575 22976 tablet_service.cc:1514] Processing DeleteTablet for tablet 246e5c2d8644461cb3b71b2acdf85bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:47:22 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:44042
I20260808 10:47:22.751663 22770 tablet_service.cc:1514] Processing DeleteTablet for tablet 246e5c2d8644461cb3b71b2acdf85bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:47:22 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:41002
I20260808 10:47:22.751745 22576 tablet_service.cc:1514] Processing DeleteTablet for tablet 246e5c2d8644461cb3b71b2acdf85bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:47:22 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:43110
I20260808 10:47:22.752234 23129 tablet_replica.cc:330] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6: stopping tablet replica
I20260808 10:47:22.752310 23130 tablet_replica.cc:330] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f: stopping tablet replica
I20260808 10:47:22.752373 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 22456
I20260808 10:47:22.752388 23129 raft_consensus.cc:2229] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:47:22.752518 23129 raft_consensus.cc:2258] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:47:22.752789 23128 tablet_replica.cc:330] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc: stopping tablet replica
I20260808 10:47:22.754926 23130 raft_consensus.cc:2229] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:47:22.755121 23128 raft_consensus.cc:2229] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:47:22.755268 23130 raft_consensus.cc:2258] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:47:22.755476 23128 raft_consensus.cc:2258] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:47:22.757252 23128 ts_tablet_manager.cc:1905] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:47:22.760280 23128 ts_tablet_manager.cc:1918] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:47:22.760447 23128 log.cc:1196] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/246e5c2d8644461cb3b71b2acdf85bd7
I20260808 10:47:22.760829 23128 ts_tablet_manager.cc:1939] T 246e5c2d8644461cb3b71b2acdf85bd7 P d709d62000ff4579a6bf93923e440cdc: Deleting consensus metadata
I20260808 10:47:22.763309 22389 catalog_manager.cc:4841] TS d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973): tablet 246e5c2d8644461cb3b71b2acdf85bd7 (table test-table [id=88ad4249a80240b683f38079093e8ad7]) successfully deleted
I20260808 10:47:22.767213 23130 ts_tablet_manager.cc:1905] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:47:22.770498 23130 ts_tablet_manager.cc:1918] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:47:22.770627 23130 log.cc:1196] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/246e5c2d8644461cb3b71b2acdf85bd7
I20260808 10:47:22.770967 23130 ts_tablet_manager.cc:1939] T 246e5c2d8644461cb3b71b2acdf85bd7 P 59a4550bf97a4de6a4c1ea827be7e81f: Deleting consensus metadata
I20260808 10:47:22.772388 22388 catalog_manager.cc:4841] TS 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977): tablet 246e5c2d8644461cb3b71b2acdf85bd7 (table test-table [id=88ad4249a80240b683f38079093e8ad7]) successfully deleted
W20260808 10:47:22.781513 22386 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:37879 (error 108)
I20260808 10:47:22.782254 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 22673
I20260808 10:47:22.782931 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:22.782463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:37879 (local address 127.0.0.1:37774)
0808 10:47:22.782727 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:22.782821 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:37879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260808 10:47:22.783119 22386 catalog_manager.cc:4584] TS c53d5237a75840849a3ce6d7f10445d6 (127.31.237.129:37879): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 246e5c2d8644461cb3b71b2acdf85bd7: Network error: Client connection negotiation failed: client connection to 127.31.237.129:37879: connect: Connection refused (error 111)
I20260808 10:47:22.798393 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 22868
I20260808 10:47:22.810230 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 22358
I20260808 10:47:22.827230 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:34147
--webserver_interface=127.31.237.190
--webserver_port=35951
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:34147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:22.944344 23132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:22.944608 23132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:22.944684 23132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:22.948743 23132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:47:22.948803 23132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:22.948823 23132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:22.948839 23132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:47:22.948890 23132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:47:22.954396 23132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:34147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:34147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=35951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:22.955523 23132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:22.956602 23132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:22.965839 23145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:22.965942 23141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:22.965960 23132 server_base.cc:1034] running on GCE node
W20260808 10:47:22.966192 23142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:22.966825 23132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:22.968003 23132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:22.969205 23132 hybrid_clock.cc:648] HybridClock initialized: now 1786186042969188 us; error 32 us; skew 500 ppm
Aug 08 10:47:22 dist-test-slave-twwt krb5kdc[21564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186042, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:22.972203 23132 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:47:22.973165 23132 webserver.cc:460] Webserver started at http://127.31.237.190:35951/ using document root <none> and password file <none>
I20260808 10:47:22.973706 23132 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:22.973767 23132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:22.977716 23132 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:47:22.980067 23153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:22.981344 23132 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:47:22.981458 23132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "0e53e08be91e402aba993003729aca96"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "4e50087ef5cae52dfb4378f3e11d7456"
server_key_iv: "e34f8b61c7ffa7ff8d9a30f0d005218a"
server_key_version: "encryptionkey@0"
I20260808 10:47:22.982018 23132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:23.000025 23132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:23.003059 23132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:23.003271 23132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:23.010959 23132 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:34147
I20260808 10:47:23.011269 23236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:34147 every 8 connection(s)
I20260808 10:47:23.012261 23132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:47:23.014400 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 23132
I20260808 10:47:23.014948 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:37879
--local_ip_for_outbound_sockets=127.31.237.129
--tserver_master_addrs=127.31.237.190:34147
--webserver_port=43729
--webserver_interface=127.31.237.129
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:47:23.018039 23237 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:47:23.019949 23237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Bootstrap starting.
I20260808 10:47:23.038902 23237 log.cc:824] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:23.051203 23237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:47:23.051645 23237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96: Bootstrap complete.
I20260808 10:47:23.056061 23237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } }
I20260808 10:47:23.056486 23237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e53e08be91e402aba993003729aca96, State: Initialized, Role: FOLLOWER
I20260808 10:47:23.057055 23237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } }
I20260808 10:47:23.057181 23237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:47:23.057281 23237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:47:23.057420 23237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 2 FOLLOWER]: Advancing to term 3
I20260808 10:47:23.059540 23237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } }
I20260808 10:47:23.059928 23237 leader_election.cc:302] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e53e08be91e402aba993003729aca96; no voters:
I20260808 10:47:23.060315 23237 leader_election.cc:288] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 10:47:23.060420 23241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 10:47:23.060642 23241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [term 3 LEADER]: Becoming Leader. State: Replica: 0e53e08be91e402aba993003729aca96, State: Running, Role: LEADER
I20260808 10:47:23.061061 23241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } }
I20260808 10:47:23.061697 23237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:47:23.062296 23241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e53e08be91e402aba993003729aca96. Latest consensus state: current_term: 3 leader_uuid: "0e53e08be91e402aba993003729aca96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } } }
I20260808 10:47:23.062435 23241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:23.062882 23248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:47:23.063004 23243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0e53e08be91e402aba993003729aca96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e53e08be91e402aba993003729aca96" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34147 } } }
I20260808 10:47:23.063199 23243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e53e08be91e402aba993003729aca96 [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:23.068498 23248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:47:23.069321 23248 catalog_manager.cc:1259] Loaded cluster ID: 12232d378e5541e395c50311b32274b8
I20260808 10:47:23.069399 23248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 10:47:23.070573 23248 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbc45cb79ad google::LogMessage::Fail() at ??:0
@ 0x7fbc45cba88a google::LogMessage::SendToLog() at ??:0
@ 0x7fbc45cb74de google::LogMessage::Flush() at ??:0
@ 0x7fbc45cb8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbc505fafdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fbc505fbd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbc505f8579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fbc5063c640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fbc47f093d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbc47f09c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fbc47f0b3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fbc47efa512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbc4953e6db start_thread at ??:0
@ 0x7fbc447b561f clone at ??:0
W20260808 10:47:23.159433 23239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:23.159883 23239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:23.159960 23239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:23.166720 23239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:23.166815 23239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:23.166926 23239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:47:23.175091 23239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:37879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=43729
--tserver_master_addrs=127.31.237.190:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:23.176535 23239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:23.178017 23239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:23.190930 23289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:23.191172 23297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:23.190908 23295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:23.191361 23239 server_base.cc:1034] running on GCE node
I20260808 10:47:23.191804 23239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:23.192512 23239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:23.193717 23239 hybrid_clock.cc:648] HybridClock initialized: now 1786186043193706 us; error 52 us; skew 500 ppm
Aug 08 10:47:23 dist-test-slave-twwt krb5kdc[21564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186043, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:23.196884 23239 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:47:23.197897 23239 webserver.cc:460] Webserver started at http://127.31.237.129:43729/ using document root <none> and password file <none>
I20260808 10:47:23.198519 23239 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:23.198643 23239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:23.202872 23239 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:47:23.205256 23307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:23.206664 23239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:47:23.206816 23239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "c53d5237a75840849a3ce6d7f10445d6"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "3c8e45c0e5552c52e3f1eca8da8242db"
server_key_iv: "2124f5265c2f0d1b19e5ef08c35017d8"
server_key_version: "encryptionkey@0"
I20260808 10:47:23.207386 23239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:23.238405 23239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:23.241971 23239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:23.242256 23239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:23.243031 23239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:23.244324 23317 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:47:23.250319 23239 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:47:23.250420 23239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 10:47:23.250535 23239 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:47:23.252317 23239 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:47:23.252401 23239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 10:47:23.252709 23317 tablet_bootstrap.cc:492] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6: Bootstrap starting.
I20260808 10:47:23.263705 23239 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:37879
I20260808 10:47:23.264038 23470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:37879 every 8 connection(s)
I20260808 10:47:23.264801 23239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:47:23.274304 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 23239
I20260808 10:47:23.274822 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:34977
--local_ip_for_outbound_sockets=127.31.237.130
--tserver_master_addrs=127.31.237.190:34147
--webserver_port=38361
--webserver_interface=127.31.237.130
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:47:23.275719 23317 log.cc:824] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:23.283217 23317 tablet_bootstrap.cc:492] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:47:23.283603 23317 tablet_bootstrap.cc:492] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6: Bootstrap complete.
I20260808 10:47:23.284924 23317 ts_tablet_manager.cc:1397] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6: Time spent bootstrapping tablet: real 0.032s user 0.025s sys 0.004s
I20260808 10:47:23.287423 23317 raft_consensus.cc:348] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:23.287878 23317 raft_consensus.cc:729] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c53d5237a75840849a3ce6d7f10445d6, State: Initialized, Role: FOLLOWER
I20260808 10:47:23.288379 23317 consensus_queue.cc:260] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:23.289333 23317 ts_tablet_manager.cc:1428] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
W20260808 10:47:23.339231 23475 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:47:23.266952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:39137)
0808 10:47:23.267456 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:23.267510 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:23.268344 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:23.338248 (+ 69904us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":141,"threads_started":1}
W20260808 10:47:23.339962 23472 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:34147 (0 consecutive failures): Network error: Failed to ping master at 127.31.237.190:34147: Client connection negotiation failed: client connection to 127.31.237.190:34147: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:47:23.340816 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:23.340487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:34629)
0808 10:47:23.340629 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:23.340713 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 10:47:23.341625 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:23.341383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:55147)
0808 10:47:23.341526 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:23.341575 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 10:47:23.341974 23472 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:47:23.412587 23478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:23.412890 23478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:23.412976 23478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:23.417498 23478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:23.417569 23478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:23.417672 23478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:47:23.423213 23478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:34977
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=38361
--tserver_master_addrs=127.31.237.190:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:23.424286 23478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:23.425344 23478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:23.435334 23489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:23.435299 23491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:23.435323 23488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:23.435474 23478 server_base.cc:1034] running on GCE node
I20260808 10:47:23.435974 23478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:23.436650 23478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:23.437867 23478 hybrid_clock.cc:648] HybridClock initialized: now 1786186043437852 us; error 35 us; skew 500 ppm
Aug 08 10:47:23 dist-test-slave-twwt krb5kdc[21564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186043, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:23.440659 23478 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:47:23.441571 23478 webserver.cc:460] Webserver started at http://127.31.237.130:38361/ using document root <none> and password file <none>
I20260808 10:47:23.442174 23478 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:23.442268 23478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:23.446112 23478 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:47:23.448323 23504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:23.449642 23478 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:47:23.449800 23478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "59a4550bf97a4de6a4c1ea827be7e81f"
format_stamp: "Formatted at 2026-08-08 10:47:20 on dist-test-slave-twwt"
server_key: "7a2a147f820e869dc60dd50ba8006685"
server_key_iv: "5b6b784ee0c5a90ca50766dd36d4ad24"
server_key_version: "encryptionkey@0"
I20260808 10:47:23.450397 23478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:23.469393 23478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:23.472570 23478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:23.472803 23478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:23.473440 23478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:23.474308 23478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:23.474383 23478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:23.474457 23478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:23.474506 23478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:23.484257 23478 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:34977
I20260808 10:47:23.484284 23680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:34977 every 8 connection(s)
I20260808 10:47:23.485311 23478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:47:23.489097 23683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:23.487482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:53449)
0808 10:47:23.487949 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:23.488384 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":104,"threads_started":1}
W20260808 10:47:23.489574 23681 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:34147 (0 consecutive failures): Network error: Failed to ping master at 127.31.237.190:34147: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
I20260808 10:47:23.490163 23683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:23.489957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:57125)
0808 10:47:23.490064 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:23.490111 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 10:47:23.490916 23683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:23.490729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:52623)
0808 10:47:23.490824 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:23.490871 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 10:47:23.491149 23681 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:47:23.493947 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 23478
I20260808 10:47:23.494410 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:39973
--local_ip_for_outbound_sockets=127.31.237.131
--tserver_master_addrs=127.31.237.190:34147
--webserver_port=46323
--webserver_interface=127.31.237.131
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:23.614507 23685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:23.614935 23685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:23.615034 23685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:23.619441 23685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:23.619525 23685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:23.619617 23685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:47:23.625015 23685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:39973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=46323
--tserver_master_addrs=127.31.237.190:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:23.626173 23685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:23.627315 23685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:23.637390 23692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:23.637406 23691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:23.637396 23694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:23.637744 23685 server_base.cc:1034] running on GCE node
I20260808 10:47:23.638170 23685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:23.638881 23685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:23.640048 23685 hybrid_clock.cc:648] HybridClock initialized: now 1786186043640023 us; error 35 us; skew 500 ppm
Aug 08 10:47:23 dist-test-slave-twwt krb5kdc[21564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186043, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:23.643180 23685 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:47:23.644079 23685 webserver.cc:460] Webserver started at http://127.31.237.131:46323/ using document root <none> and password file <none>
I20260808 10:47:23.644626 23685 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:23.644690 23685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:23.648602 23685 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:47:23.650851 23705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:23.652316 23685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:47:23.652427 23685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "d709d62000ff4579a6bf93923e440cdc"
format_stamp: "Formatted at 2026-08-08 10:47:21 on dist-test-slave-twwt"
server_key: "ad1142777267a1b3b6eb7cba12810e0e"
server_key_iv: "2924571fcf10922815afeb0f24f395ba"
server_key_version: "encryptionkey@0"
I20260808 10:47:23.653004 23685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:23.672334 23685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:23.675609 23685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:23.675770 23685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:23.676347 23685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:23.677229 23685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:23.677289 23685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:23.677352 23685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:23.677376 23685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:23.686695 23685 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:39973
I20260808 10:47:23.686740 23876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:39973 every 8 connection(s)
I20260808 10:47:23.687824 23685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:47:23.691954 23881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:23.690074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:53941)
0808 10:47:23.690596 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:23.691168 (+ 572us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":122,"threads_started":1}
I20260808 10:47:23.692416 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 23685
W20260808 10:47:23.692613 23877 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:34147 (0 consecutive failures): Network error: Failed to ping master at 127.31.237.190:34147: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
I20260808 10:47:23.693363 23881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:23.693073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:50835)
0808 10:47:23.693222 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:23.693290 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 10:47:23.694064 23881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:23.693821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:56145)
0808 10:47:23.693961 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:23.694009 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260808 10:47:23.694355 23877 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:47:24.343218 23883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:24.342723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:49861)
0808 10:47:24.342980 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:24.343034 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":74,"threads_started":1}
I20260808 10:47:24.492097 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:24.491698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:42005)
0808 10:47:24.491938 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:24.491996 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":70,"threads_started":1}
I20260808 10:47:24.553864 23886 raft_consensus.cc:482] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:47:24.554098 23886 raft_consensus.cc:504] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:24.555397 23886 leader_election.cc:288] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973), 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977)
Aug 08 10:47:24 dist-test-slave-twwt krb5kdc[21564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186043, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.130@KRBTEST.COM
Aug 08 10:47:24 dist-test-slave-twwt krb5kdc[21564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186043, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.131@KRBTEST.COM
I20260808 10:47:24.575551 23889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:24.555812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:49525 (local address 127.31.237.131:39973)
0808 10:47:24.556272 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:24.556285 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:24.556321 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:47:24.557006 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:24.557012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:24.557449 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:24.557756 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:24.559022 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:24.559995 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:24.561224 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:24.561881 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:24.562041 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:24.570551 (+ 8510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:24.570557 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:24.570570 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:24.570604 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:24.572934 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:24.573430 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:24.573436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:24.573442 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:24.573573 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:24.573881 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:24.573885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:24.573887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:24.574311 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:24.574462 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:24.575071 (+ 609us) server_negotiation.cc:299] Negotiation successful
0808 10:47:24.575352 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":104,"threads_started":1}
I20260808 10:47:24.575541 23883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:24.555599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:39973 (local address 127.31.237.129:49525)
0808 10:47:24.555813 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:24.555828 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:24.555945 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:24.557808 (+ 1863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:24.557815 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:24.558330 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:24.558818 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:24.558833 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:24.560159 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:24.560168 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:24.561092 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:24.561102 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:24.562001 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:24.562004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:24.562214 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:24.568201 (+ 5987us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:24.568223 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:24.570291 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:24.573058 (+ 2767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:24.573061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:24.573063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:24.573274 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:24.573681 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:24.573684 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:24.573686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:24.573774 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:24.574427 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:24.574430 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:24.574744 (+ 314us) client_negotiation.cc:769] Sending connection context
0808 10:47:24.574949 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 10:47:24.575337 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":195,"spinlock_wait_cycles":1280}
I20260808 10:47:24.575583 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:24.555813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:54151 (local address 127.31.237.130:34977)
0808 10:47:24.556318 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:24.556339 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:24.556380 (+ 41us) server_negotiation.cc:407] Connection header received
0808 10:47:24.557031 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:24.557037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:24.557463 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:24.557720 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:24.559017 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:24.559992 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:24.561258 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:24.561915 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:24.562102 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:24.570404 (+ 8302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:24.570411 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:24.570427 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:24.570462 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:24.572539 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:24.573277 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:24.573283 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:24.573290 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:24.573436 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:24.573807 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:24.573811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:24.573813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:24.574279 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:24.574449 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:24.575073 (+ 624us) server_negotiation.cc:299] Negotiation successful
0808 10:47:24.575382 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":103,"threads_started":1}
I20260808 10:47:24.576028 23888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:24.555628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:34977 (local address 127.31.237.129:54151)
0808 10:47:24.556047 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:24.556062 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:24.556152 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:24.558098 (+ 1946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:24.558101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:24.558330 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:24.558817 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:24.558833 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:24.560151 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:24.560155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:24.561112 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:24.561125 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:24.562049 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:24.562053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:24.562195 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:24.568202 (+ 6007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:24.568232 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:24.570029 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:24.572749 (+ 2720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:24.572757 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:24.572774 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:24.573139 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:24.573555 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:24.573562 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:24.573564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:24.573701 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:24.574419 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:24.574426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:24.574744 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 10:47:24.574949 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 10:47:24.575431 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":294,"spinlock_wait_cycles":10624,"thread_start_us":80,"threads_started":1}
W20260808 10:47:24.578316 23314 leader_election.cc:341] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973): Not found: Tablet not found: 246e5c2d8644461cb3b71b2acdf85bd7
W20260808 10:47:24.578464 23313 leader_election.cc:341] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977): Not found: Tablet not found: 246e5c2d8644461cb3b71b2acdf85bd7
I20260808 10:47:24.578588 23313 leader_election.cc:302] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c53d5237a75840849a3ce6d7f10445d6; no voters: 59a4550bf97a4de6a4c1ea827be7e81f, d709d62000ff4579a6bf93923e440cdc
I20260808 10:47:24.578922 23886 raft_consensus.cc:2726] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:47:24.695271 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:24.694901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:34511)
0808 10:47:24.695126 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:24.695177 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
I20260808 10:47:25.344476 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:25.344048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:47077)
0808 10:47:25.344316 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:25.344375 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":78,"threads_started":1}
I20260808 10:47:25.493396 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:25.492931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:58669)
0808 10:47:25.493228 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:25.493292 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":105,"threads_started":1}
I20260808 10:47:25.696527 23900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:25.696062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:40431)
0808 10:47:25.696338 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:25.696409 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":83,"threads_started":1}
I20260808 10:47:26.256986 23901 raft_consensus.cc:482] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:47:26.257126 23901 raft_consensus.cc:504] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:26.257599 23901 leader_election.cc:288] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973), 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977)
W20260808 10:47:26.258496 23314 leader_election.cc:341] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973): Not found: Tablet not found: 246e5c2d8644461cb3b71b2acdf85bd7
W20260808 10:47:26.258595 23313 leader_election.cc:341] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977): Not found: Tablet not found: 246e5c2d8644461cb3b71b2acdf85bd7
I20260808 10:47:26.258728 23313 leader_election.cc:302] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c53d5237a75840849a3ce6d7f10445d6; no voters: 59a4550bf97a4de6a4c1ea827be7e81f, d709d62000ff4579a6bf93923e440cdc
I20260808 10:47:26.258885 23901 raft_consensus.cc:2726] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:47:26.345701 23902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:26.345263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:48727)
0808 10:47:26.345535 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:26.345593 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":98,"threads_started":1}
I20260808 10:47:26.494568 23903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:26.494144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:56993)
0808 10:47:26.494418 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:26.494476 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":89,"threads_started":1}
I20260808 10:47:26.697922 23905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:26.697415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:48733)
0808 10:47:26.697690 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:26.697758 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":77,"threads_started":1}
I20260808 10:47:27.346952 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:27.346482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:52245)
0808 10:47:27.346790 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:27.346845 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":82,"threads_started":1}
I20260808 10:47:27.495770 23910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:27.495364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:55253)
0808 10:47:27.495621 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:27.495676 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":76,"threads_started":1}
I20260808 10:47:27.699314 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:27.698866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:53317)
0808 10:47:27.699145 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:27.699211 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
I20260808 10:47:27.945331 23912 raft_consensus.cc:482] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:47:27.945479 23912 raft_consensus.cc:504] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:27.945869 23912 leader_election.cc:288] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973), 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977)
W20260808 10:47:27.946738 23313 leader_election.cc:341] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977): Not found: Tablet not found: 246e5c2d8644461cb3b71b2acdf85bd7
W20260808 10:47:27.946838 23314 leader_election.cc:341] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973): Not found: Tablet not found: 246e5c2d8644461cb3b71b2acdf85bd7
I20260808 10:47:27.946938 23314 leader_election.cc:302] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c53d5237a75840849a3ce6d7f10445d6; no voters: 59a4550bf97a4de6a4c1ea827be7e81f, d709d62000ff4579a6bf93923e440cdc
I20260808 10:47:27.947124 23912 raft_consensus.cc:2726] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:47:28.348470 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:28.347950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:34281)
0808 10:47:28.348301 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:28.348364 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":99,"threads_started":1}
I20260808 10:47:28.497112 23915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:28.496673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:40827)
0808 10:47:28.496941 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:28.497008 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":82,"threads_started":1}
I20260808 10:47:28.700595 23916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:28.700131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:48479)
0808 10:47:28.700421 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:28.700484 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":94,"threads_started":1}
I20260808 10:47:29.349709 23917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:29.349219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:37447)
0808 10:47:29.349543 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:29.349607 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":110,"threads_started":1}
I20260808 10:47:29.498219 23920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:29.497835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:36055)
0808 10:47:29.498075 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:29.498127 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":75,"threads_started":1}
I20260808 10:47:29.678148 23921 raft_consensus.cc:482] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:47:29.678292 23921 raft_consensus.cc:504] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:29.678725 23921 leader_election.cc:288] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973), 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977)
W20260808 10:47:29.679656 23313 leader_election.cc:341] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977): Not found: Tablet not found: 246e5c2d8644461cb3b71b2acdf85bd7
W20260808 10:47:29.679772 23314 leader_election.cc:341] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973): Not found: Tablet not found: 246e5c2d8644461cb3b71b2acdf85bd7
I20260808 10:47:29.679834 23314 leader_election.cc:302] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c53d5237a75840849a3ce6d7f10445d6; no voters: 59a4550bf97a4de6a4c1ea827be7e81f, d709d62000ff4579a6bf93923e440cdc
I20260808 10:47:29.680022 23921 raft_consensus.cc:2726] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:47:29.702034 23922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:29.701576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:46797)
0808 10:47:29.701868 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:29.701932 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":97,"threads_started":1}
I20260808 10:47:30.350971 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:30.350508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:51411)
0808 10:47:30.350796 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:30.350869 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":123,"threads_started":1}
I20260808 10:47:30.499682 23924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:30.499205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:52287)
0808 10:47:30.499516 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:30.499581 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":74,"threads_started":1}
I20260808 10:47:30.703408 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:30.702927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:43103)
0808 10:47:30.703241 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:30.703305 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":73,"threads_started":1}
I20260808 10:47:31.352161 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:31.351746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:56731)
0808 10:47:31.352000 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:31.352057 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":82,"threads_started":1}
I20260808 10:47:31.501078 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:31.500610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:40051)
0808 10:47:31.500900 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:31.500977 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":92,"threads_started":1}
I20260808 10:47:31.704622 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:31.704198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:48507)
0808 10:47:31.704466 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:31.704525 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":95,"threads_started":1}
I20260808 10:47:32.353564 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:32.353088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:54295)
0808 10:47:32.353398 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:32.353461 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":117,"threads_started":1}
I20260808 10:47:32.502341 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:32.501884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:60243)
0808 10:47:32.502179 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:32.502242 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":83,"threads_started":1}
I20260808 10:47:32.705838 23934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:32.705440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:47461)
0808 10:47:32.705687 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:32.705741 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":74,"threads_started":1}
I20260808 10:47:33.354924 23936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:33.354449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:47891)
0808 10:47:33.354754 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:33.354816 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":83,"threads_started":1}
I20260808 10:47:33.503690 23937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:33.503272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:59267)
0808 10:47:33.503538 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:33.503593 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":89,"threads_started":1}
I20260808 10:47:33.693486 23938 raft_consensus.cc:482] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:47:33.693629 23938 raft_consensus.cc:504] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d709d62000ff4579a6bf93923e440cdc" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 39973 } } peers { permanent_uuid: "59a4550bf97a4de6a4c1ea827be7e81f" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34977 } } peers { permanent_uuid: "c53d5237a75840849a3ce6d7f10445d6" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37879 } }
I20260808 10:47:33.694054 23938 leader_election.cc:288] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973), 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977)
W20260808 10:47:33.695005 23314 leader_election.cc:341] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d709d62000ff4579a6bf93923e440cdc (127.31.237.131:39973): Not found: Tablet not found: 246e5c2d8644461cb3b71b2acdf85bd7
W20260808 10:47:33.695132 23313 leader_election.cc:341] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59a4550bf97a4de6a4c1ea827be7e81f (127.31.237.130:34977): Not found: Tablet not found: 246e5c2d8644461cb3b71b2acdf85bd7
I20260808 10:47:33.695261 23313 leader_election.cc:302] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c53d5237a75840849a3ce6d7f10445d6; no voters: 59a4550bf97a4de6a4c1ea827be7e81f, d709d62000ff4579a6bf93923e440cdc
I20260808 10:47:33.695431 23938 raft_consensus.cc:2726] T 246e5c2d8644461cb3b71b2acdf85bd7 P c53d5237a75840849a3ce6d7f10445d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:47:33.707103 23940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:33.706689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:60577)
0808 10:47:33.706951 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:33.707009 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":99,"threads_started":1}
I20260808 10:47:34.356187 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:34.355730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:57621)
0808 10:47:34.356016 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:34.356082 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":80,"threads_started":1}
I20260808 10:47:34.505043 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:34.504523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:49603)
0808 10:47:34.504846 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:34.504902 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":125,"threads_started":1}
I20260808 10:47:34.708484 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:34.707864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:60601)
0808 10:47:34.708236 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:34.708338 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":80,"threads_started":1}
I20260808 10:47:35.357734 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:35.357167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:47419)
0808 10:47:35.357491 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:35.357614 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":101,"threads_started":1}
I20260808 10:47:35.506336 23947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:35.505908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:37677)
0808 10:47:35.506176 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:35.506232 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":75,"threads_started":1}
I20260808 10:47:35.709959 23948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:35.709340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:34735)
0808 10:47:35.709720 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:35.709820 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":99,"threads_started":1}
I20260808 10:47:36.359180 23949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:36.358722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:55711)
0808 10:47:36.359014 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:36.359080 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":82,"threads_started":1}
I20260808 10:47:36.507532 23951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:36.507119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:39153)
0808 10:47:36.507377 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:36.507434 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
I20260808 10:47:36.711351 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:36.710887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:52829)
0808 10:47:36.711186 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:36.711248 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":106,"threads_started":1}
I20260808 10:47:37.360412 23953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:37.359959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:34911)
0808 10:47:37.360231 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:37.360293 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":78,"threads_started":1}
I20260808 10:47:37.508771 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:37.508345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:35513)
0808 10:47:37.508613 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:37.508671 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":87,"threads_started":1}
I20260808 10:47:37.712695 23956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:37.712223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:41731)
0808 10:47:37.712518 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:37.712586 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":97,"threads_started":1}
I20260808 10:47:38.361768 23957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:38.361356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.129:36787)
0808 10:47:38.361611 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:38.361666 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":90,"threads_started":1}
I20260808 10:47:38.510257 23959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:38.509806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.130:48507)
0808 10:47:38.510094 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:38.510154 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":100,"threads_started":1}
I20260808 10:47:38.696882 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 23239
I20260808 10:47:38.711802 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 23478
I20260808 10:47:38.720827 23960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:38.720367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34147 (local address 127.31.237.131:35269)
0808 10:47:38.720661 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:38.720725 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:34147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":103,"threads_started":1}
I20260808 10:47:38.726369 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 23685
2026-08-08T10:47:38Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19865 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:47:38 dist-test-slave-twwt krb5kdc[23967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:47:38 dist-test-slave-twwt krb5kdc[23967](info): set up 2 sockets
Aug 08 10:47:38 dist-test-slave-twwt krb5kdc[23967](info): commencing operation
krb5kdc: starting...
W20260808 10:47:39.979787 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:47:40 dist-test-slave-twwt krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:47:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:47:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:47:40.113616 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:33319
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:33319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:40.230412 23984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:40.230744 23984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:40.230840 23984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:40.234804 23984 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:47:40.234884 23984 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:47:40.234912 23984 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:47:40.234942 23984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:47:40.234985 23984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:40.235025 23984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:40.235057 23984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:47:40.235090 23984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:47:40.240232 23984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:33319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:33319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:40.241302 23984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:40.242344 23984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:47:40.251392 23984 server_base.cc:1034] running on GCE node
W20260808 10:47:40.251405 23994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:40.251405 23995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:40.251570 23997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:40.252295 23984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:40.253463 23984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:40.254670 23984 hybrid_clock.cc:648] HybridClock initialized: now 1786186060254648 us; error 29 us; skew 500 ppm
Aug 08 10:47:40 dist-test-slave-twwt krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186060, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:40.257790 23984 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:47:40.258865 23984 webserver.cc:460] Webserver started at http://127.31.237.190:44463/ using document root <none> and password file <none>
I20260808 10:47:40.259451 23984 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:40.259543 23984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:40.259804 23984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:47:40.261900 23984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "0245337fc0694a92ae345f8d88be22aa"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "df375d2c88868456420f9954a8221257"
server_key_iv: "cd39205438d6e65dd3e5a1659140623c"
server_key_version: "encryptionkey@0"
I20260808 10:47:40.262491 23984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "0245337fc0694a92ae345f8d88be22aa"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "df375d2c88868456420f9954a8221257"
server_key_iv: "cd39205438d6e65dd3e5a1659140623c"
server_key_version: "encryptionkey@0"
I20260808 10:47:40.266278 23984 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:47:40.268889 24010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:40.270159 23984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:47:40.270303 23984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "0245337fc0694a92ae345f8d88be22aa"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "df375d2c88868456420f9954a8221257"
server_key_iv: "cd39205438d6e65dd3e5a1659140623c"
server_key_version: "encryptionkey@0"
I20260808 10:47:40.270416 23984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:40.287322 23984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:40.290752 23984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:40.290966 23984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:40.298468 23984 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:33319
I20260808 10:47:40.298842 24101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:33319 every 8 connection(s)
I20260808 10:47:40.299603 23984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:47:40.300818 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 23984
I20260808 10:47:40.300959 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:47:40.301266 32694 external_mini_cluster.cc:1428] Setting key f51d7706a2acae7c6825b37e8208387d
I20260808 10:47:40.302920 24102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:40.309814 24102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Bootstrap starting.
Aug 08 10:47:40 dist-test-slave-twwt krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:40.312341 24102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:40.313314 24102 log.cc:824] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:40.315739 24102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: No bootstrap required, opened a new log
I20260808 10:47:40.316668 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:40.302708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36074 (local address 127.31.237.190:33319)
0808 10:47:40.303232 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:40.303253 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:40.303308 (+ 55us) server_negotiation.cc:407] Connection header received
0808 10:47:40.304151 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:40.304176 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:40.304600 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:40.304964 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:40.305667 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:40.307225 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:40.308067 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:40.308776 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:40.308941 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:40.311476 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:40.311483 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:40.311498 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:40.311529 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:40.313575 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:40.314081 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:40.314086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:40.314090 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:40.314238 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:40.314517 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:40.314520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:40.314521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:40.315045 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:40.315235 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:40.315445 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 10:47:40.315734 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":151,"threads_started":1}
I20260808 10:47:40.319821 24102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } }
I20260808 10:47:40.320029 24102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:40.320092 24102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0245337fc0694a92ae345f8d88be22aa, State: Initialized, Role: FOLLOWER
I20260808 10:47:40.320523 24102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } }
I20260808 10:47:40.320680 24102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:47:40.320775 24102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:47:40.320909 24102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:40.321949 24102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } }
I20260808 10:47:40.322292 24102 leader_election.cc:302] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0245337fc0694a92ae345f8d88be22aa; no voters:
I20260808 10:47:40.322592 24102 leader_election.cc:288] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:47:40.322767 24108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:47:40.323002 24108 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 1 LEADER]: Becoming Leader. State: Replica: 0245337fc0694a92ae345f8d88be22aa, State: Running, Role: LEADER
I20260808 10:47:40.323374 24108 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } }
I20260808 10:47:40.323843 24102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:47:40.324965 24111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0245337fc0694a92ae345f8d88be22aa. Latest consensus state: current_term: 1 leader_uuid: "0245337fc0694a92ae345f8d88be22aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } } }
I20260808 10:47:40.325109 24111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:40.325538 24121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:47:40.325489 24109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0245337fc0694a92ae345f8d88be22aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } } }
I20260808 10:47:40.326150 24109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: This master's current role is: LEADER
I20260808 10:47:40.328931 24121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:47:40.334436 24121 catalog_manager.cc:1347] Generated new cluster ID: 24e3c81fd1c944f099581943be50e9b8
I20260808 10:47:40.334501 24121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:47:40.363096 24121 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:47:40.364142 24121 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:47:40.371572 24121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Generated new TSK 0
I20260808 10:47:40.372434 24121 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:47:40.413272 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33319
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:40.527029 24145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:40.527274 24145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:40.527316 24145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:40.531291 24145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:40.531345 24145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:40.531427 24145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:47:40.536466 24145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:40.537444 24145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:40.538426 24145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:40.548113 24154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:40.548081 24152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:40.548200 24145 server_base.cc:1034] running on GCE node
W20260808 10:47:40.548097 24151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:40.548725 24145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:40.549502 24145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:40.550688 24145 hybrid_clock.cc:648] HybridClock initialized: now 1786186060550661 us; error 32 us; skew 500 ppm
Aug 08 10:47:40 dist-test-slave-twwt krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186060, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:40.553617 24145 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:47:40.554483 24145 webserver.cc:460] Webserver started at http://127.31.237.129:35217/ using document root <none> and password file <none>
I20260808 10:47:40.555066 24145 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:40.555132 24145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:40.555311 24145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:47:40.557304 24145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "1c398392519d48cbbbc8aea1c8d2dbdf"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "57621fe0eaa74412bfc5b1a288c31e9a"
server_key_iv: "27ade598afe73dcca52fecc9c0e70547"
server_key_version: "encryptionkey@0"
I20260808 10:47:40.557849 24145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "1c398392519d48cbbbc8aea1c8d2dbdf"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "57621fe0eaa74412bfc5b1a288c31e9a"
server_key_iv: "27ade598afe73dcca52fecc9c0e70547"
server_key_version: "encryptionkey@0"
I20260808 10:47:40.561693 24145 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:47:40.564170 24162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:40.565407 24145 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:47:40.565514 24145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "1c398392519d48cbbbc8aea1c8d2dbdf"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "57621fe0eaa74412bfc5b1a288c31e9a"
server_key_iv: "27ade598afe73dcca52fecc9c0e70547"
server_key_version: "encryptionkey@0"
I20260808 10:47:40.565594 24145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:40.584293 24145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:40.587684 24145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:40.587843 24145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:40.588390 24145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:40.589190 24145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:40.589236 24145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:40.589277 24145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:40.589293 24145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:40.598873 24145 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:43819
I20260808 10:47:40.598995 24337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:43819 every 8 connection(s)
I20260808 10:47:40.599846 24145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:47:40.600168 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 24145
I20260808 10:47:40.600309 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:47:40.600662 32694 external_mini_cluster.cc:1428] Setting key 7d4835cac08d6e3895ef9b88a2e934b0
Aug 08 10:47:40 dist-test-slave-twwt krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186060, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:47:40.614075 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:40.601795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:60851 (local address 127.31.237.190:33319)
0808 10:47:40.601951 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:40.601954 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:40.602989 (+ 1035us) server_negotiation.cc:407] Connection header received
0808 10:47:40.604315 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:40.604318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:40.604372 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:40.604457 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:40.605623 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:40.606150 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:40.607362 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:40.607617 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:40.607729 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:40.610584 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:40.610587 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:40.610589 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:40.610649 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:40.612165 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:40.612700 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:40.612703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:40.612704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:40.612786 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:40.613101 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:40.613103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:40.613104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:40.613304 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:40.613378 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:40.613844 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 10:47:40.613953 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:47:40.614974 24341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:40.602247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:60851)
0808 10:47:40.602834 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:40.602887 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:40.604109 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:40.604634 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:40.604641 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:40.605007 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:40.605480 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:40.605493 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:40.606284 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:40.606290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:40.607237 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:40.607248 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:40.607773 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:40.607777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:40.607953 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:40.608599 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:40.608616 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:40.610409 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:40.612281 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:40.612290 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:40.612310 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:40.612611 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:40.612879 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:40.612882 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:40.612883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:40.613018 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:40.613401 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:40.613407 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:40.613641 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 10:47:40.613799 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 10:47:40.614065 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":149,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
I20260808 10:47:40.616266 24338 heartbeater.cc:347] Connected to a master server at 127.31.237.190:33319
I20260808 10:47:40.616539 24338 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:40.617142 24338 heartbeater.cc:511] Master 127.31.237.190:33319 requested a full tablet report, sending...
I20260808 10:47:40.618896 24043 ts_manager.cc:194] Registered new tserver with Master: 1c398392519d48cbbbc8aea1c8d2dbdf (127.31.237.129:43819)
I20260808 10:47:40.620136 24043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:60851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:47:40.633373 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33319
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:40.750080 24344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:40.750375 24344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:40.750458 24344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:40.754426 24344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:40.754515 24344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:40.754626 24344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:47:40.759905 24344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:40.761042 24344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:40.762118 24344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:40.772851 24359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:40.773020 24354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:40.773267 24344 server_base.cc:1034] running on GCE node
W20260808 10:47:40.772850 24357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:40.774039 24344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:40.774873 24344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:40.776145 24344 hybrid_clock.cc:648] HybridClock initialized: now 1786186060776106 us; error 52 us; skew 500 ppm
Aug 08 10:47:40 dist-test-slave-twwt krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186060, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:40.779314 24344 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:47:40.780293 24344 webserver.cc:460] Webserver started at http://127.31.237.130:38005/ using document root <none> and password file <none>
I20260808 10:47:40.780896 24344 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:40.780990 24344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:40.781242 24344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:47:40.783489 24344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "527f950bf42e45f9940ed23462ab8a57"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "f07de554a51a5b47abdea857f2d128c4"
server_key_iv: "44bd9bcf7c55b93dce3c633ebff7cd87"
server_key_version: "encryptionkey@0"
I20260808 10:47:40.784065 24344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "527f950bf42e45f9940ed23462ab8a57"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "f07de554a51a5b47abdea857f2d128c4"
server_key_iv: "44bd9bcf7c55b93dce3c633ebff7cd87"
server_key_version: "encryptionkey@0"
I20260808 10:47:40.788046 24344 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:47:40.790584 24377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:40.792168 24344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:47:40.792279 24344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "527f950bf42e45f9940ed23462ab8a57"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "f07de554a51a5b47abdea857f2d128c4"
server_key_iv: "44bd9bcf7c55b93dce3c633ebff7cd87"
server_key_version: "encryptionkey@0"
I20260808 10:47:40.792372 24344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:40.842083 24344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:40.845628 24344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:40.845834 24344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:40.846446 24344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:40.847352 24344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:40.847399 24344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:40.847440 24344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:40.847456 24344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:40.856673 24344 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:33803
I20260808 10:47:40.856700 24537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:33803 every 8 connection(s)
I20260808 10:47:40.857573 24344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:47:40.861428 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 24344
I20260808 10:47:40.861563 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:47:40.861886 32694 external_mini_cluster.cc:1428] Setting key da57cf7e8f30716d81f4827dd8fb02ee
Aug 08 10:47:40 dist-test-slave-twwt krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186060, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:47:40.873359 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:40.859421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:42695 (local address 127.31.237.190:33319)
0808 10:47:40.859577 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:40.859582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:40.860497 (+ 915us) server_negotiation.cc:407] Connection header received
0808 10:47:40.861903 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:40.861907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:40.861972 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:40.862071 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:40.863680 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:40.864443 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:40.865660 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:40.865890 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:40.866006 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:40.868994 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:40.868997 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:40.868999 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:40.869024 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:40.870710 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:40.871389 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:40.871392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:40.871393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:40.871474 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:40.871934 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:40.871937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:40.871938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:40.872137 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:40.872236 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:40.872956 (+ 720us) server_negotiation.cc:299] Negotiation successful
0808 10:47:40.873120 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:47:40.874261 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:40.859858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:42695)
0808 10:47:40.860337 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:40.860383 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:40.861645 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:40.862226 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:40.862233 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:40.862842 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:40.863516 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:40.863546 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:40.864585 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:40.864588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:40.865523 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:40.865533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:40.866050 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:40.866054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:40.866243 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:40.866900 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:40.866917 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:40.868808 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:40.870857 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:40.870867 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:40.870887 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:40.871269 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:40.871576 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:40.871581 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:40.871608 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:40.871806 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:40.872278 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:40.872287 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:40.872644 (+ 357us) client_negotiation.cc:769] Sending connection context
0808 10:47:40.872907 (+ 263us) client_negotiation.cc:240] Negotiation successful
0808 10:47:40.873285 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":119,"threads_started":1}
I20260808 10:47:40.875563 24539 heartbeater.cc:347] Connected to a master server at 127.31.237.190:33319
I20260808 10:47:40.875866 24539 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:40.876441 24539 heartbeater.cc:511] Master 127.31.237.190:33319 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
I20260808 10:47:40.877667 24043 ts_manager.cc:194] Registered new tserver with Master: 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803)
I20260808 10:47:40.878535 24043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:42695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:47:40.894188 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33319
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:47:41.013949 24548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:47:41.014284 24548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:47:41.014369 24548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:47:41.018440 24548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:47:41.018548 24548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:47:41.018666 24548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:47:41.024276 24548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:47:41.025467 24548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:47:41.026579 24548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:47:41.036710 24563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:41.036671 24566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:47:41.036808 24562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:47:41.037128 24548 server_base.cc:1034] running on GCE node
I20260808 10:47:41.037592 24548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:47:41.038306 24548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:47:41.039499 24548 hybrid_clock.cc:648] HybridClock initialized: now 1786186061039458 us; error 61 us; skew 500 ppm
Aug 08 10:47:41 dist-test-slave-twwt krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186061, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:47:41.042459 24548 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:47:41.043448 24548 webserver.cc:460] Webserver started at http://127.31.237.131:44045/ using document root <none> and password file <none>
I20260808 10:47:41.044072 24548 fs_manager.cc:359] Metadata directory not provided
I20260808 10:47:41.044135 24548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:47:41.044310 24548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:47:41.046386 24548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "941a0de7d1364e8293277b9514cbf7ff"
format_stamp: "Formatted at 2026-08-08 10:47:41 on dist-test-slave-twwt"
server_key: "15f91363b32b97db4b129c6cc00b54bd"
server_key_iv: "c6a7cb7ac6cd22f87fbbc3b57a919d15"
server_key_version: "encryptionkey@0"
I20260808 10:47:41.047001 24548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "941a0de7d1364e8293277b9514cbf7ff"
format_stamp: "Formatted at 2026-08-08 10:47:41 on dist-test-slave-twwt"
server_key: "15f91363b32b97db4b129c6cc00b54bd"
server_key_iv: "c6a7cb7ac6cd22f87fbbc3b57a919d15"
server_key_version: "encryptionkey@0"
I20260808 10:47:41.050802 24548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:47:41.053375 24575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:41.054702 24548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:47:41.054824 24548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "941a0de7d1364e8293277b9514cbf7ff"
format_stamp: "Formatted at 2026-08-08 10:47:41 on dist-test-slave-twwt"
server_key: "15f91363b32b97db4b129c6cc00b54bd"
server_key_iv: "c6a7cb7ac6cd22f87fbbc3b57a919d15"
server_key_version: "encryptionkey@0"
I20260808 10:47:41.054958 24548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:47:41.065670 24548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:47:41.069144 24548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:47:41.069372 24548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:47:41.069986 24548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:47:41.070926 24548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:47:41.070997 24548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:41.071110 24548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:47:41.071153 24548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:47:41.080696 24548 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:41991
I20260808 10:47:41.080717 24740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:41991 every 8 connection(s)
I20260808 10:47:41.081701 24548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:47:41.082007 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 24548
I20260808 10:47:41.082162 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:47:41.082522 32694 external_mini_cluster.cc:1428] Setting key 3fd339499901bdf16138b646ea217e97
Aug 08 10:47:41 dist-test-slave-twwt krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186061, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:47:41.096163 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.083866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:43937 (local address 127.31.237.190:33319)
0808 10:47:41.084035 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:41.084040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:41.085052 (+ 1012us) server_negotiation.cc:407] Connection header received
0808 10:47:41.086394 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:41.086397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:41.086452 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:41.086532 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:41.087769 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.088312 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.089513 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.089744 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.089859 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:41.092458 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:41.092461 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:41.092463 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:41.092488 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:41.093833 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:41.094318 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:41.094320 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:41.094322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:41.094400 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:41.094807 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:41.094809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:41.094810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:41.095004 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:41.095083 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:41.095867 (+ 784us) server_negotiation.cc:299] Negotiation successful
0808 10:47:41.096020 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:47:41.097150 24743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.084377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:43937)
0808 10:47:41.084898 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:41.084959 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:41.086195 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:41.086754 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:41.086761 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:41.087119 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:47:41.087626 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:41.087640 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.088450 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.088453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:41.089369 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:41.089379 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.089902 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.089907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:41.090090 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:41.090686 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:47:41.090702 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:47:41.092269 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:47:41.093945 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:41.093950 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:41.093963 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:41.094227 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:41.094497 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:47:41.094500 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:47:41.094501 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:47:41.094721 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:47:41.095121 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:47:41.095128 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:47:41.095473 (+ 345us) client_negotiation.cc:769] Sending connection context
0808 10:47:41.095679 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 10:47:41.096064 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":121,"threads_started":1}
I20260808 10:47:41.098359 24741 heartbeater.cc:347] Connected to a master server at 127.31.237.190:33319
I20260808 10:47:41.098654 24741 heartbeater.cc:464] Registering TS with master...
I20260808 10:47:41.099200 24741 heartbeater.cc:511] Master 127.31.237.190:33319 requested a full tablet report, sending...
I20260808 10:47:41.100231 24043 ts_manager.cc:194] Registered new tserver with Master: 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991)
I20260808 10:47:41.100816 24043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:43937
I20260808 10:47:41.108603 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:47:41.118317 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.110679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36090 (local address 127.31.237.190:33319)
0808 10:47:41.110803 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:41.110807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:41.111051 (+ 244us) server_negotiation.cc:407] Connection header received
0808 10:47:41.111151 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:41.111153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:41.111191 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:41.111260 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:47:41.111947 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.112465 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.113296 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.113531 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.113622 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:41.115029 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:47:41.115033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:47:41.115035 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:47:41.115058 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:47:41.116654 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:41.117049 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:41.117051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:41.117052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:41.117128 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:47:41.117382 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:47:41.117385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:47:41.117386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:47:41.117610 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:47:41.117714 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:41.118063 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 10:47:41.118176 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:47:41.121374 24043 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:47:41.124856 24043 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:47:41.140260 24767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.135027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51566 (local address 127.31.237.131:41991)
0808 10:47:41.135458 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:41.135471 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:41.135809 (+ 338us) server_negotiation.cc:407] Connection header received
0808 10:47:41.135886 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:41.135890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:41.136056 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:41.136182 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:41.136607 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.137323 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.138894 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.139529 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.139655 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:41.139791 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:41.140002 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 10:47:41.140104 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":141,"threads_started":1}
I20260808 10:47:41.140250 24761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.134538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:41991 (local address 127.0.0.1:51566)
0808 10:47:41.135614 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:41.135628 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:41.135718 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:41.136324 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:41.136326 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:41.136352 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:41.136479 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:41.136486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.137464 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.137471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:41.138773 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:41.138783 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.139659 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.139663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:41.139791 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:41.139835 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 10:47:41.139951 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 10:47:41.140077 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1012,"mutex_wait_us":31,"spinlock_wait_cycles":2688,"thread_start_us":116,"threads_started":1}
I20260808 10:47:41.141006 24760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.134053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:33803 (local address 127.0.0.1:35682)
0808 10:47:41.134926 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:41.134979 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:41.135118 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:41.136289 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:41.136294 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:41.136316 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:41.136457 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:41.136465 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.137842 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.137845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:41.139294 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:41.139305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.140589 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.140593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:41.140700 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:41.140725 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:47:41.140805 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 10:47:41.140876 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":134,"spinlock_wait_cycles":1408,"thread_start_us":95,"threads_started":1}
I20260808 10:47:41.141557 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.134252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:43819 (local address 127.0.0.1:56968)
0808 10:47:41.135485 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:41.135500 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:41.135602 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:41.136082 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:41.136088 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:41.136111 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:41.136266 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:41.136273 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.137763 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.137769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:41.139294 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:41.139305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.140942 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.140946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:41.141034 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:41.141284 (+ 250us) client_negotiation.cc:769] Sending connection context
0808 10:47:41.141354 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 10:47:41.141421 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1159,"mutex_wait_us":137,"thread_start_us":89,"threads_started":1}
I20260808 10:47:41.141769 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.134279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.31.237.130:33803)
0808 10:47:41.135555 (+ 1276us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:41.135571 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:41.135595 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:47:41.135685 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:41.135691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:41.135946 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:41.136103 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:41.136587 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.137702 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.139451 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.140261 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.140388 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:41.140520 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:41.141322 (+ 802us) server_negotiation.cc:299] Negotiation successful
0808 10:47:41.141539 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1178,"thread_start_us":230,"threads_started":1}
I20260808 10:47:41.142000 24766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.134954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56968 (local address 127.31.237.129:43819)
0808 10:47:41.135351 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:41.135359 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:41.135675 (+ 316us) server_negotiation.cc:407] Connection header received
0808 10:47:41.135775 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:41.135779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:41.135930 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:41.136092 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:41.136387 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.137612 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.139433 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.140234 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.140720 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:41.140865 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:41.141738 (+ 873us) server_negotiation.cc:299] Negotiation successful
0808 10:47:41.141845 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":85,"threads_started":1}
I20260808 10:47:41.143937 24642 tablet_service.cc:1467] Processing CreateTablet for tablet b07fa1e3ccae488e9a763e274d174c1b (DEFAULT_TABLE table=test-table [id=6698e28cd9b44d8b9bb71e4d1464ca96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:47:41.144476 24240 tablet_service.cc:1467] Processing CreateTablet for tablet b07fa1e3ccae488e9a763e274d174c1b (DEFAULT_TABLE table=test-table [id=6698e28cd9b44d8b9bb71e4d1464ca96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:47:41.144631 24441 tablet_service.cc:1467] Processing CreateTablet for tablet b07fa1e3ccae488e9a763e274d174c1b (DEFAULT_TABLE table=test-table [id=6698e28cd9b44d8b9bb71e4d1464ca96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:47:41.145438 24642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b07fa1e3ccae488e9a763e274d174c1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:41.145474 24240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b07fa1e3ccae488e9a763e274d174c1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:41.145568 24441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b07fa1e3ccae488e9a763e274d174c1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:47:41.151822 24768 tablet_bootstrap.cc:492] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: Bootstrap starting.
I20260808 10:47:41.153625 24773 tablet_bootstrap.cc:492] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff: Bootstrap starting.
I20260808 10:47:41.154332 24768 tablet_bootstrap.cc:654] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:41.154968 24770 tablet_bootstrap.cc:492] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57: Bootstrap starting.
I20260808 10:47:41.155527 24768 log.cc:824] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:41.155898 24773 tablet_bootstrap.cc:654] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:41.157191 24773 log.cc:824] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:41.157498 24770 tablet_bootstrap.cc:654] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57: Neither blocks nor log segments found. Creating new log.
I20260808 10:47:41.158239 24768 tablet_bootstrap.cc:492] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: No bootstrap required, opened a new log
I20260808 10:47:41.158439 24768 ts_tablet_manager.cc:1397] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260808 10:47:41.158447 24770 log.cc:824] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57: Log is configured to *not* fsync() on all Append() calls
I20260808 10:47:41.159297 24773 tablet_bootstrap.cc:492] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff: No bootstrap required, opened a new log
I20260808 10:47:41.159497 24773 ts_tablet_manager.cc:1397] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260808 10:47:41.160238 24770 tablet_bootstrap.cc:492] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57: No bootstrap required, opened a new log
I20260808 10:47:41.160442 24770 ts_tablet_manager.cc:1397] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:47:41.163556 24768 raft_consensus.cc:348] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } } peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } }
I20260808 10:47:41.163556 24773 raft_consensus.cc:348] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } } peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } }
I20260808 10:47:41.163848 24768 raft_consensus.cc:374] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:41.163848 24773 raft_consensus.cc:374] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:41.163686 24770 raft_consensus.cc:348] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } } peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } }
I20260808 10:47:41.163908 24773 raft_consensus.cc:729] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 941a0de7d1364e8293277b9514cbf7ff, State: Initialized, Role: FOLLOWER
I20260808 10:47:41.163908 24768 raft_consensus.cc:729] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c398392519d48cbbbc8aea1c8d2dbdf, State: Initialized, Role: FOLLOWER
I20260808 10:47:41.163950 24770 raft_consensus.cc:374] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:47:41.164009 24770 raft_consensus.cc:729] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 527f950bf42e45f9940ed23462ab8a57, State: Initialized, Role: FOLLOWER
I20260808 10:47:41.164383 24768 consensus_queue.cc:260] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } } peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } }
I20260808 10:47:41.164389 24773 consensus_queue.cc:260] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } } peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } }
I20260808 10:47:41.164425 24770 consensus_queue.cc:260] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } } peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } }
I20260808 10:47:41.165134 24338 heartbeater.cc:503] Master 127.31.237.190:33319 was elected leader, sending a full tablet report...
I20260808 10:47:41.165436 24741 heartbeater.cc:503] Master 127.31.237.190:33319 was elected leader, sending a full tablet report...
I20260808 10:47:41.165851 24539 heartbeater.cc:503] Master 127.31.237.190:33319 was elected leader, sending a full tablet report...
I20260808 10:47:41.165460 24773 ts_tablet_manager.cc:1428] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260808 10:47:41.166525 24768 ts_tablet_manager.cc:1428] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260808 10:47:41.167343 24770 ts_tablet_manager.cc:1428] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260808 10:47:41.265084 24778 raft_consensus.cc:482] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:47:41.265390 24778 raft_consensus.cc:504] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } } peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } }
I20260808 10:47:41.266762 24778 leader_election.cc:288] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803), 1c398392519d48cbbbc8aea1c8d2dbdf (127.31.237.129:43819)
I20260808 10:47:41.272193 24766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.267250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:54947 (local address 127.31.237.129:43819)
0808 10:47:41.267386 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:41.267391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:41.267789 (+ 398us) server_negotiation.cc:407] Connection header received
0808 10:47:41.267886 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:41.267888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:41.267930 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:41.267990 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:41.268277 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.269014 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.270768 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.271411 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.271534 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:41.271589 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:41.271984 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 10:47:41.272061 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:47:41.272193 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.267216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:52933 (local address 127.31.237.130:33803)
0808 10:47:41.267381 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:41.267386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:41.267437 (+ 51us) server_negotiation.cc:407] Connection header received
0808 10:47:41.267505 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:41.267508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:41.267582 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:41.267665 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:47:41.268098 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.268931 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.270769 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.271404 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.271534 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:41.271589 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:41.271984 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 10:47:41.272059 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 10:47:41.272254 24743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.267066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:33803 (local address 127.31.237.131:52933)
0808 10:47:41.267229 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:41.267246 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:41.267365 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:41.267711 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:41.267771 (+ 60us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:41.267798 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:41.267945 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:41.267951 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.269097 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.269102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:41.270624 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:41.270637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.271547 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.271551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:41.271814 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:41.271860 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 10:47:41.271946 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:47:41.272087 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":211,"spinlock_wait_cycles":3200}
I20260808 10:47:41.272455 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.267066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:43819 (local address 127.31.237.131:54947)
0808 10:47:41.267669 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:41.267683 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:47:41.267798 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:47:41.268026 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:47:41.268031 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:47:41.268055 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:47:41.268157 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:41.268161 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.269110 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.269112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:41.270629 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:47:41.270637 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.271547 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.271551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:47:41.271698 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:41.271867 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 10:47:41.271946 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 10:47:41.272087 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":482,"spinlock_wait_cycles":5888,"thread_start_us":145,"threads_started":1}
I20260808 10:47:41.273159 24477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b07fa1e3ccae488e9a763e274d174c1b" candidate_uuid: "941a0de7d1364e8293277b9514cbf7ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "527f950bf42e45f9940ed23462ab8a57" is_pre_election: true
I20260808 10:47:41.273214 24266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b07fa1e3ccae488e9a763e274d174c1b" candidate_uuid: "941a0de7d1364e8293277b9514cbf7ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" is_pre_election: true
I20260808 10:47:41.273536 24477 raft_consensus.cc:2454] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 941a0de7d1364e8293277b9514cbf7ff in term 0.
I20260808 10:47:41.273536 24266 raft_consensus.cc:2454] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 941a0de7d1364e8293277b9514cbf7ff in term 0.
I20260808 10:47:41.274073 24583 leader_election.cc:302] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c398392519d48cbbbc8aea1c8d2dbdf, 941a0de7d1364e8293277b9514cbf7ff; no voters:
I20260808 10:47:41.274420 24778 raft_consensus.cc:2781] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:47:41.274505 24778 raft_consensus.cc:482] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:47:41.274537 24778 raft_consensus.cc:3037] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:41.275691 24778 raft_consensus.cc:504] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } } peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } }
I20260808 10:47:41.276062 24778 leader_election.cc:288] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [CANDIDATE]: Term 1 election: Requested vote from peers 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803), 1c398392519d48cbbbc8aea1c8d2dbdf (127.31.237.129:43819)
I20260808 10:47:41.276418 24477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b07fa1e3ccae488e9a763e274d174c1b" candidate_uuid: "941a0de7d1364e8293277b9514cbf7ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "527f950bf42e45f9940ed23462ab8a57"
I20260808 10:47:41.276559 24266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b07fa1e3ccae488e9a763e274d174c1b" candidate_uuid: "941a0de7d1364e8293277b9514cbf7ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf"
I20260808 10:47:41.276575 24477 raft_consensus.cc:3037] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:41.276690 24266 raft_consensus.cc:3037] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:47:41.277995 24477 raft_consensus.cc:2454] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 941a0de7d1364e8293277b9514cbf7ff in term 1.
I20260808 10:47:41.277995 24266 raft_consensus.cc:2454] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 941a0de7d1364e8293277b9514cbf7ff in term 1.
I20260808 10:47:41.278380 24576 leader_election.cc:302] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 527f950bf42e45f9940ed23462ab8a57, 941a0de7d1364e8293277b9514cbf7ff; no voters:
I20260808 10:47:41.278657 24778 raft_consensus.cc:2781] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:47:41.278927 24778 raft_consensus.cc:686] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [term 1 LEADER]: Becoming Leader. State: Replica: 941a0de7d1364e8293277b9514cbf7ff, State: Running, Role: LEADER
I20260808 10:47:41.279378 24778 consensus_queue.cc:237] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } } peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } }
I20260808 10:47:41.282841 24043 catalog_manager.cc:5495] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131). New cstate: current_term: 1 leader_uuid: "941a0de7d1364e8293277b9514cbf7ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } health_report { overall_health: HEALTHY } } }
I20260808 10:47:41.330264 24767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.326545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51580 (local address 127.31.237.131:41991)
0808 10:47:41.326712 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 10:47:41.326716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:47:41.326732 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:47:41.326962 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:47:41.326965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:47:41.327025 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:47:41.327140 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:47:41.327404 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.327908 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.328796 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:47:41.329019 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:47:41.329132 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:47:41.329452 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:47:41.329876 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:47:41.330011 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 10:47:41.330053 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 10:47:41.330112 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 10:47:41.332876 24742 tablet.cc:2369] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:47:41.336746 24477 raft_consensus.cc:1261] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57 [term 1 FOLLOWER]: Refusing update from remote peer 941a0de7d1364e8293277b9514cbf7ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:47:41.336746 24266 raft_consensus.cc:1261] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Refusing update from remote peer 941a0de7d1364e8293277b9514cbf7ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:47:41.337507 24786 consensus_queue.cc:1035] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:47:41.337664 24778 consensus_queue.cc:1035] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:47:41.346002 24792 mvcc.cc:205] Tried to move back new op lower bound from 7316218107227467776 to 7316218107003351040. Current Snapshot: MvccSnapshot[applied={T|T < 7316218107227467776}]
I20260808 10:47:41.346829 24793 mvcc.cc:205] Tried to move back new op lower bound from 7316218107227467776 to 7316218107003351040. Current Snapshot: MvccSnapshot[applied={T|T < 7316218107227467776}]
I20260808 10:47:41.356153 24043 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:47:41.356340 24043 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:47:41.359144 24043 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Sending DeleteTablet for 3 replicas of tablet b07fa1e3ccae488e9a763e274d174c1b
I20260808 10:47:41.359937 24441 tablet_service.cc:1514] Processing DeleteTablet for tablet b07fa1e3ccae488e9a763e274d174c1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:47:41 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:35682
I20260808 10:47:41.360030 24642 tablet_service.cc:1514] Processing DeleteTablet for tablet b07fa1e3ccae488e9a763e274d174c1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:47:41 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:51566
I20260808 10:47:41.360303 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 24145
I20260808 10:47:41.367847 24803 tablet_replica.cc:330] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57: stopping tablet replica
I20260808 10:47:41.368292 24803 raft_consensus.cc:2229] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:47:41.368654 24804 tablet_replica.cc:330] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff: stopping tablet replica
I20260808 10:47:41.368750 24803 raft_consensus.cc:2258] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:47:41.368996 24804 raft_consensus.cc:2229] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:47:41.369452 24804 raft_consensus.cc:2258] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:47:41.370580 24803 ts_tablet_manager.cc:1905] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:47:41.370949 24804 ts_tablet_manager.cc:1905] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:47:41.373678 24803 ts_tablet_manager.cc:1918] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:47:41.373679 24804 ts_tablet_manager.cc:1918] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:47:41.373824 24804 log.cc:1196] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/b07fa1e3ccae488e9a763e274d174c1b
I20260808 10:47:41.373839 24803 log.cc:1196] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/b07fa1e3ccae488e9a763e274d174c1b
I20260808 10:47:41.374221 24803 ts_tablet_manager.cc:1939] T b07fa1e3ccae488e9a763e274d174c1b P 527f950bf42e45f9940ed23462ab8a57: Deleting consensus metadata
I20260808 10:47:41.374305 24804 ts_tablet_manager.cc:1939] T b07fa1e3ccae488e9a763e274d174c1b P 941a0de7d1364e8293277b9514cbf7ff: Deleting consensus metadata
I20260808 10:47:41.375705 24016 catalog_manager.cc:4841] TS 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991): tablet b07fa1e3ccae488e9a763e274d174c1b (table test-table [id=6698e28cd9b44d8b9bb71e4d1464ca96]) successfully deleted
I20260808 10:47:41.375989 24012 catalog_manager.cc:4841] TS 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803): tablet b07fa1e3ccae488e9a763e274d174c1b (table test-table [id=6698e28cd9b44d8b9bb71e4d1464ca96]) successfully deleted
W20260808 10:47:41.377167 24017 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:43819 (error 108)
I20260808 10:47:41.377488 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 24344
I20260808 10:47:41.384349 24761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:47:41.384016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:43819 (local address 127.0.0.1:56984)
0808 10:47:41.384155 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 10:47:41.384248 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 10:47:41.384645 24017 catalog_manager.cc:4584] TS 1c398392519d48cbbbc8aea1c8d2dbdf (127.31.237.129:43819): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b07fa1e3ccae488e9a763e274d174c1b: Network error: Client connection negotiation failed: client connection to 127.31.237.129:43819: connect: Connection refused (error 111)
I20260808 10:47:41.391263 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 24548
I20260808 10:47:41.403148 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 23984
I20260808 10:48:01.417371 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:33319
--webserver_interface=127.31.237.190
--webserver_port=44463
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:33319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:01.545569 24806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:01.545868 24806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:01.545914 24806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:01.550496 24806 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:48:01.550558 24806 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:48:01.550576 24806 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:48:01.550592 24806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:48:01.550683 24806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:01.550727 24806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:01.550760 24806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:48:01.550793 24806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:48:01.556730 24806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:33319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:33319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=44463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:01.557871 24806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:01.559001 24806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:01.575245 24815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:01.575245 24818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:01.575351 24806 server_base.cc:1034] running on GCE node
W20260808 10:48:01.575245 24816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:01.576526 24806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:01.577944 24806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:01.579180 24806 hybrid_clock.cc:648] HybridClock initialized: now 1786186081579141 us; error 55 us; skew 500 ppm
Aug 08 10:48:01 dist-test-slave-twwt krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:01.588449 24806 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:48:01.589638 24806 webserver.cc:460] Webserver started at http://127.31.237.190:44463/ using document root <none> and password file <none>
I20260808 10:48:01.590313 24806 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:01.590441 24806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:01.595041 24806 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 10:48:01.597709 24826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:01.599363 24806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:48:01.599550 24806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "0245337fc0694a92ae345f8d88be22aa"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "df375d2c88868456420f9954a8221257"
server_key_iv: "cd39205438d6e65dd3e5a1659140623c"
server_key_version: "encryptionkey@0"
I20260808 10:48:01.600270 24806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:01.614657 24806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:01.618446 24806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:01.618714 24806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:01.626824 24806 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:33319
I20260808 10:48:01.627168 24917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:33319 every 8 connection(s)
I20260808 10:48:01.628201 24806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:48:01.633163 24918 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:48:01.634996 24918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Bootstrap starting.
I20260808 10:48:01.637697 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 24806
I20260808 10:48:01.638200 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:43819
--local_ip_for_outbound_sockets=127.31.237.129
--tserver_master_addrs=127.31.237.190:33319
--webserver_port=35217
--webserver_interface=127.31.237.129
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:48:01.653594 24918 log.cc:824] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:01.663151 24918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:48:01.663595 24918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Bootstrap complete.
I20260808 10:48:01.668018 24918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } }
I20260808 10:48:01.668435 24918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0245337fc0694a92ae345f8d88be22aa, State: Initialized, Role: FOLLOWER
I20260808 10:48:01.668974 24918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } }
I20260808 10:48:01.669098 24918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:48:01.669199 24918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:48:01.669327 24918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:48:01.671558 24918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } }
I20260808 10:48:01.671959 24918 leader_election.cc:302] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0245337fc0694a92ae345f8d88be22aa; no voters:
I20260808 10:48:01.672331 24918 leader_election.cc:288] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:48:01.672436 24924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:48:01.672672 24924 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 2 LEADER]: Becoming Leader. State: Replica: 0245337fc0694a92ae345f8d88be22aa, State: Running, Role: LEADER
I20260808 10:48:01.673031 24924 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } }
I20260808 10:48:01.673748 24918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:48:01.674515 24925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0245337fc0694a92ae345f8d88be22aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } } }
I20260808 10:48:01.674686 24925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:01.675402 24933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:48:01.675447 24926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0245337fc0694a92ae345f8d88be22aa. Latest consensus state: current_term: 2 leader_uuid: "0245337fc0694a92ae345f8d88be22aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } } }
I20260808 10:48:01.675577 24926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:01.680856 24933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:48:01.681699 24933 catalog_manager.cc:1259] Loaded cluster ID: 24e3c81fd1c944f099581943be50e9b8
I20260808 10:48:01.681766 24933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:48:01.684356 24933 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:48:01.685067 24933 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Loaded TSK: 0
I20260808 10:48:01.699160 24933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Generated new TSK 1
I20260808 10:48:01.699528 24933 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:48:01.769035 24921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:01.769315 24921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:01.769361 24921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:01.774498 24921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:01.774588 24921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:01.774732 24921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:48:01.780666 24921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:43819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=35217
--tserver_master_addrs=127.31.237.190:33319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:01.781908 24921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:01.783125 24921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:01.794785 24956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:01.794811 24958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:01.794788 24961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:01.795189 24921 server_base.cc:1034] running on GCE node
I20260808 10:48:01.795778 24921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:01.796651 24921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:01.797881 24921 hybrid_clock.cc:648] HybridClock initialized: now 1786186081797852 us; error 38 us; skew 500 ppm
Aug 08 10:48:01 dist-test-slave-twwt krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:01.801481 24921 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:48:01.802531 24921 webserver.cc:460] Webserver started at http://127.31.237.129:35217/ using document root <none> and password file <none>
I20260808 10:48:01.803164 24921 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:01.803277 24921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:01.807791 24921 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:48:01.810304 24970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:01.811656 24921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:48:01.811805 24921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "1c398392519d48cbbbc8aea1c8d2dbdf"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "57621fe0eaa74412bfc5b1a288c31e9a"
server_key_iv: "27ade598afe73dcca52fecc9c0e70547"
server_key_version: "encryptionkey@0"
I20260808 10:48:01.812381 24921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:01.831593 24921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:01.835342 24921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:01.835569 24921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:01.836228 24921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:01.837360 24981 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:48:01.841529 24921 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:48:01.841620 24921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260808 10:48:01.841706 24921 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:48:01.843532 24921 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:48:01.843611 24921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 10:48:01.843926 24981 tablet_bootstrap.cc:492] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: Bootstrap starting.
I20260808 10:48:01.855238 24921 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:43819
I20260808 10:48:01.855296 25125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:43819 every 8 connection(s)
I20260808 10:48:01.856208 24921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:48:01.857008 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 24921
I20260808 10:48:01.857450 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:33803
--local_ip_for_outbound_sockets=127.31.237.130
--tserver_master_addrs=127.31.237.190:33319
--webserver_port=38005
--webserver_interface=127.31.237.130
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:48:01.866091 24981 log.cc:824] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:48:01 dist-test-slave-twwt krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:48:01.875973 24981 tablet_bootstrap.cc:492] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:48:01.876428 24981 tablet_bootstrap.cc:492] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: Bootstrap complete.
I20260808 10:48:01.878050 24981 ts_tablet_manager.cc:1397] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: Time spent bootstrapping tablet: real 0.034s user 0.021s sys 0.009s
I20260808 10:48:01.879053 25131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:01.859712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:49205 (local address 127.31.237.190:33319)
0808 10:48:01.860226 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:01.860247 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:01.860296 (+ 49us) server_negotiation.cc:407] Connection header received
0808 10:48:01.860801 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:01.860812 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:01.861116 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:01.861452 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:01.862634 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:01.863642 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:01.864894 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:01.865393 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:01.865592 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:01.873162 (+ 7570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:01.873168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:01.873181 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:01.873213 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:01.875469 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:01.876168 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:01.876173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:01.876178 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:01.876302 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:01.876720 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:01.876724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:01.876726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:01.877095 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:01.877253 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:01.877815 (+ 562us) server_negotiation.cc:299] Negotiation successful
0808 10:48:01.878100 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":105,"threads_started":1}
I20260808 10:48:01.879053 25129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:01.858572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:49205)
0808 10:48:01.859214 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:01.859280 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:01.860355 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:01.861505 (+ 1150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:01.861512 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:01.861949 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:01.862425 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:01.862438 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:01.863874 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:01.863877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:01.864751 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:01.864761 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:01.865535 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:01.865539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:01.865730 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:01.871306 (+ 5576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:01.871325 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:01.872900 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:01.875628 (+ 2728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:01.875634 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:01.875648 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:01.875924 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:01.876420 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:01.876424 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:01.876426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:01.876609 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:01.877236 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:01.877243 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:01.877496 (+ 253us) client_negotiation.cc:769] Sending connection context
0808 10:48:01.877693 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 10:48:01.878083 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":173,"threads_started":1}
I20260808 10:48:01.881695 25126 heartbeater.cc:347] Connected to a master server at 127.31.237.190:33319
I20260808 10:48:01.881314 24981 raft_consensus.cc:348] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } } peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } }
I20260808 10:48:01.881867 24981 raft_consensus.cc:729] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c398392519d48cbbbc8aea1c8d2dbdf, State: Initialized, Role: FOLLOWER
I20260808 10:48:01.881949 25126 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:01.882390 24981 consensus_queue.cc:260] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } } peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } }
I20260808 10:48:01.882548 25126 heartbeater.cc:511] Master 127.31.237.190:33319 requested a full tablet report, sending...
I20260808 10:48:01.883494 24981 ts_tablet_manager.cc:1428] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260808 10:48:01.885097 24854 ts_manager.cc:194] Registered new tserver with Master: 1c398392519d48cbbbc8aea1c8d2dbdf (127.31.237.129:43819)
I20260808 10:48:01.888880 24854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:49205
I20260808 10:48:01.891454 25126 heartbeater.cc:503] Master 127.31.237.190:33319 was elected leader, sending a full tablet report...
Aug 08 10:48:01 dist-test-slave-twwt krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186081, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for kudu/127.31.237.129@KRBTEST.COM
I20260808 10:48:01.900134 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:01.888216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:43819 (local address 127.0.0.1:49910)
0808 10:48:01.889032 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:01.889083 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:01.889290 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:01.890003 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:01.890008 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:01.890462 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:01.890595 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:01.890628 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:01.891932 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:01.891935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:01.892869 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:01.892881 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:01.893648 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:01.893650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:01.893734 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:01.894342 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:01.894355 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:01.896064 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:01.898211 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:01.898219 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:01.898225 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:01.898489 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:01.898935 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:01.898937 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:01.898939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:01.899025 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:01.899552 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:01.899557 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:01.899663 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 10:48:01.899835 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 10:48:01.899994 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":121,"threads_started":1}
I20260808 10:48:01.900300 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:01.888337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49910 (local address 127.31.237.129:43819)
0808 10:48:01.888727 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:01.888739 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:01.889177 (+ 438us) server_negotiation.cc:407] Connection header received
0808 10:48:01.889461 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:01.889467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:01.889670 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:01.889777 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:01.890736 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:01.891784 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:01.893018 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:01.893517 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:01.893636 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:01.896223 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:01.896233 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:01.896247 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:01.896275 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:01.898079 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:01.898713 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:01.898719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:01.898721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:01.898822 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:01.899126 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:01.899129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:01.899131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:01.899430 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:01.899564 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:01.899962 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 10:48:01.900138 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":116,"threads_started":1}
I20260808 10:48:01.901341 25034 tablet_service.cc:1514] Processing DeleteTablet for tablet b07fa1e3ccae488e9a763e274d174c1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:47:41 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:49910
I20260808 10:48:01.902284 25142 tablet_replica.cc:330] stopping tablet replica
I20260808 10:48:01.902647 25142 raft_consensus.cc:2229] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:48:01.902925 25142 raft_consensus.cc:2258] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:01.904100 25142 ts_tablet_manager.cc:1905] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:01.908262 25142 ts_tablet_manager.cc:1918] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:01.908408 25142 log.cc:1196] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal/wals/b07fa1e3ccae488e9a763e274d174c1b
I20260808 10:48:01.908700 25142 ts_tablet_manager.cc:1939] T b07fa1e3ccae488e9a763e274d174c1b P 1c398392519d48cbbbc8aea1c8d2dbdf: Deleting consensus metadata
I20260808 10:48:01.910661 24833 catalog_manager.cc:4841] TS 1c398392519d48cbbbc8aea1c8d2dbdf (127.31.237.129:43819): tablet b07fa1e3ccae488e9a763e274d174c1b (table test-table [id=6698e28cd9b44d8b9bb71e4d1464ca96]) successfully deleted
W20260808 10:48:01.992885 25130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:01.993146 25130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:01.993191 25130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:01.997319 25130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:01.997381 25130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:01.997525 25130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:48:02.002823 25130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:33803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=38005
--tserver_master_addrs=127.31.237.190:33319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:02.004025 25130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:02.005165 25130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:02.015381 25153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:02.015412 25149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:02.015416 25151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:02.015779 25130 server_base.cc:1034] running on GCE node
I20260808 10:48:02.016295 25130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:02.017058 25130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:02.018270 25130 hybrid_clock.cc:648] HybridClock initialized: now 1786186082018254 us; error 28 us; skew 500 ppm
Aug 08 10:48:02 dist-test-slave-twwt krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186082, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:02.021423 25130 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:48:02.022421 25130 webserver.cc:460] Webserver started at http://127.31.237.130:38005/ using document root <none> and password file <none>
I20260808 10:48:02.023059 25130 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:02.023140 25130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:02.028067 25130 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:48:02.032187 25160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:02.034191 25130 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:48:02.034389 25130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "527f950bf42e45f9940ed23462ab8a57"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "f07de554a51a5b47abdea857f2d128c4"
server_key_iv: "44bd9bcf7c55b93dce3c633ebff7cd87"
server_key_version: "encryptionkey@0"
I20260808 10:48:02.035413 25130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:02.057145 25130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:02.061667 25130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:02.061889 25130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:02.062736 25130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:02.063843 25130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:02.063905 25130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:02.063966 25130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:02.064002 25130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:02.077075 25130 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:33803
I20260808 10:48:02.077132 25331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:33803 every 8 connection(s)
I20260808 10:48:02.078332 25130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:48:02.087823 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 25130
I20260808 10:48:02.088289 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:41991
--local_ip_for_outbound_sockets=127.31.237.131
--tserver_master_addrs=127.31.237.190:33319
--webserver_port=44045
--webserver_interface=127.31.237.131
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:48:02 dist-test-slave-twwt krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186082, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:48:02.098300 25131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.080914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:53821 (local address 127.31.237.190:33319)
0808 10:48:02.081089 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:02.081096 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:02.082187 (+ 1091us) server_negotiation.cc:407] Connection header received
0808 10:48:02.084393 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:02.084398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:02.084465 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:02.084588 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:02.086518 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.087322 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.088799 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.089059 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.089184 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.093979 (+ 4795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:02.093984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:02.093986 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:02.094017 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:02.095964 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:02.096575 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:02.096580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:02.096582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:02.096670 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:02.097055 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:02.097059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:02.097061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:02.097272 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:02.097396 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:02.097964 (+ 568us) server_negotiation.cc:299] Negotiation successful
0808 10:48:02.098118 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:48:02.099349 25338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.081260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:53821)
0808 10:48:02.081950 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:02.082019 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:02.084052 (+ 2033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:02.084832 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:02.084846 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:02.085466 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:02.086273 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:02.086297 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.087517 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.087521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:02.088657 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:02.088668 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.089244 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.089249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:02.089424 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.091722 (+ 2298us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:02.091740 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:02.093734 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:02.096116 (+ 2382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:02.096123 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:02.096143 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:02.096450 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:02.096778 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:02.096781 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:02.096783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:02.096945 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:02.097396 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:02.097405 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:02.097705 (+ 300us) client_negotiation.cc:769] Sending connection context
0808 10:48:02.097920 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 10:48:02.098310 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":145,"threads_started":1}
I20260808 10:48:02.100726 25332 heartbeater.cc:347] Connected to a master server at 127.31.237.190:33319
I20260808 10:48:02.101008 25332 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:02.101562 25332 heartbeater.cc:511] Master 127.31.237.190:33319 requested a full tablet report, sending...
I20260808 10:48:02.102761 24854 ts_manager.cc:194] Registered new tserver with Master: 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803)
I20260808 10:48:02.103541 24854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:53821
W20260808 10:48:02.215101 25341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:02.215376 25341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:02.215428 25341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:02.219705 25341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:02.219776 25341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:02.219837 25341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:48:02.225234 25341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:41991
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=44045
--tserver_master_addrs=127.31.237.190:33319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:02.226426 25341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:02.227525 25341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:48:02.238070 25341 server_base.cc:1034] running on GCE node
W20260808 10:48:02.238013 25349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:02.238011 25352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:02.238032 25350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:02.238772 25341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:02.239464 25341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:02.240810 25341 hybrid_clock.cc:648] HybridClock initialized: now 1786186082240778 us; error 40 us; skew 500 ppm
Aug 08 10:48:02 dist-test-slave-twwt krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186082, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:02.244025 25341 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:48:02.245060 25341 webserver.cc:460] Webserver started at http://127.31.237.131:44045/ using document root <none> and password file <none>
I20260808 10:48:02.245738 25341 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:02.245863 25341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:02.250150 25341 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:48:02.252498 25361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:02.253803 25341 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:48:02.253959 25341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "941a0de7d1364e8293277b9514cbf7ff"
format_stamp: "Formatted at 2026-08-08 10:47:41 on dist-test-slave-twwt"
server_key: "15f91363b32b97db4b129c6cc00b54bd"
server_key_iv: "c6a7cb7ac6cd22f87fbbc3b57a919d15"
server_key_version: "encryptionkey@0"
I20260808 10:48:02.254562 25341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:02.285089 25341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:02.288743 25341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:02.289013 25341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:02.289702 25341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:02.290654 25341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:02.290735 25341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:02.290808 25341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:02.290863 25341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:02.300493 25341 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:41991
I20260808 10:48:02.300520 25527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:41991 every 8 connection(s)
I20260808 10:48:02.301560 25341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:48:02.307228 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 25341
Aug 08 10:48:02 dist-test-slave-twwt krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186082, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:48:02.317929 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.308026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39622 (local address 127.31.237.190:33319)
0808 10:48:02.308300 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:02.308304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:02.308411 (+ 107us) server_negotiation.cc:407] Connection header received
0808 10:48:02.308511 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:02.308514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:02.308561 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:02.308660 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:02.310268 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.310977 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.312261 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.312642 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.312771 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.314254 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:02.314260 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:02.314263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:02.314330 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:02.316099 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:02.316672 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:02.316675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:02.316677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:02.316786 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:02.317086 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:02.317089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:02.317091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:02.317239 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:02.317458 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:02.317632 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 10:48:02.317738 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":108,"threads_started":1}
I20260808 10:48:02.318554 25131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.303531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:54339 (local address 127.31.237.190:33319)
0808 10:48:02.303707 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:02.303712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:02.304668 (+ 956us) server_negotiation.cc:407] Connection header received
0808 10:48:02.306164 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:02.306167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:02.306218 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:02.306294 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:02.307773 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.308575 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.309892 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.310143 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.310271 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.313604 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:02.313609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:02.313611 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:02.313635 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:02.315683 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:02.316411 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:02.316415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:02.316417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:02.316537 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:02.316988 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:02.316990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:02.316991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:02.317239 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:02.317346 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:02.318240 (+ 894us) server_negotiation.cc:299] Negotiation successful
0808 10:48:02.318371 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 10:48:02.319964 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.303947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:54339)
0808 10:48:02.304493 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:02.304555 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:02.305950 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:02.306474 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:02.306481 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:02.306993 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:02.307568 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:02.307591 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.308716 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.308719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:02.309748 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:02.309759 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.310307 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.310311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:02.310547 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.311255 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:02.311280 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:02.313391 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:02.315870 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:02.315879 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:02.315896 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:02.316270 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:02.316648 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:02.316652 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:02.316653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:02.316886 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:02.317366 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:02.317373 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:02.317760 (+ 387us) client_negotiation.cc:769] Sending connection context
0808 10:48:02.318173 (+ 413us) client_negotiation.cc:240] Negotiation successful
0808 10:48:02.318732 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":122,"threads_started":1}
I20260808 10:48:02.321164 25529 heartbeater.cc:347] Connected to a master server at 127.31.237.190:33319
I20260808 10:48:02.321473 25529 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:02.322005 25529 heartbeater.cc:511] Master 127.31.237.190:33319 requested a full tablet report, sending...
I20260808 10:48:02.323181 24854 ts_manager.cc:194] Registered new tserver with Master: 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991)
I20260808 10:48:02.323710 24854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:54339
I20260808 10:48:02.332449 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:48:02.337265 25131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.333289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39630 (local address 127.31.237.190:33319)
0808 10:48:02.333438 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:02.333444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:02.333716 (+ 272us) server_negotiation.cc:407] Connection header received
0808 10:48:02.333968 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:02.333971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:02.334084 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:02.334215 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:48:02.334483 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.335069 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.335981 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.336204 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.336321 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.336676 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:48:02.336828 (+ 152us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 10:48:02.337122 (+ 294us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39630: token expired
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 10:48:02.337364 25131 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39630: token expired
W20260808 10:48:02.337678 24746 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.237.190:33319)
I20260808 10:48:02.345988 25131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.338105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40948 (local address 127.31.237.190:33319)
0808 10:48:02.338231 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:02.338234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:02.338248 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:48:02.338359 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:02.338361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:02.338399 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:02.338466 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:02.339162 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.339632 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.340483 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.340715 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.340820 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.342294 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:02.342298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:02.342300 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:02.342322 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:02.344265 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:02.344710 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:02.344714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:02.344716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:02.344813 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:02.345106 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:02.345109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:02.345111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:02.345296 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:02.345443 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:02.345699 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 10:48:02.345822 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 10:48:02.348575 24854 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:48:02.352022 24854 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:48:02.360955 25034 tablet_service.cc:1467] Processing CreateTablet for tablet 8d77811f63cd400ab09e61d4438af1ca (DEFAULT_TABLE table=test-table [id=cd10382a60064fab8b9d46bf235bd287]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:02.361680 25034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d77811f63cd400ab09e61d4438af1ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:02.364880 24981 tablet_bootstrap.cc:492] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf: Bootstrap starting.
I20260808 10:48:02.365037 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.359638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:41991 (local address 127.0.0.1:47182)
0808 10:48:02.359778 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:02.359796 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:02.359919 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:02.360917 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:02.360920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:02.360943 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:02.361057 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:02.361066 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.362381 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.362384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:02.363399 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:02.363409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.364586 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.364590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:02.364686 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.364744 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 10:48:02.364802 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 10:48:02.364891 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 10:48:02.365748 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.359837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47182 (local address 127.31.237.131:41991)
0808 10:48:02.360370 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:02.360391 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:02.360418 (+ 27us) server_negotiation.cc:407] Connection header received
0808 10:48:02.360507 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:02.360516 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:02.360756 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:02.360937 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:02.361237 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.362249 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.363547 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.364434 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.365285 (+ 851us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.365433 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:02.365488 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 10:48:02.365597 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":163,"threads_started":1}
I20260808 10:48:02.365741 25542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.359926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:33803 (local address 127.0.0.1:50930)
0808 10:48:02.360147 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:02.360160 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:02.360250 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:02.361413 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:02.361416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:02.361436 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:02.361525 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:02.361531 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.363256 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.363260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:02.364163 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:02.364176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.365363 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.365367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:02.365454 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.365476 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:48:02.365564 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:48:02.365625 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":97,"threads_started":1}
I20260808 10:48:02.365878 24981 tablet_bootstrap.cc:654] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:02.365895 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.360363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50930 (local address 127.31.237.130:33803)
0808 10:48:02.360727 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:02.360739 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:02.360764 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:48:02.360837 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:02.360842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:02.361197 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:02.361747 (+ 550us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:02.361994 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.363123 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.364304 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.365220 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.365358 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.365492 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:02.365594 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 10:48:02.365689 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":164,"threads_started":1}
I20260808 10:48:02.367628 24981 tablet_bootstrap.cc:492] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf: No bootstrap required, opened a new log
I20260808 10:48:02.367734 24981 ts_tablet_manager.cc:1397] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:48:02.368227 24981 raft_consensus.cc:348] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:02.368357 24981 raft_consensus.cc:374] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:02.368398 24981 raft_consensus.cc:729] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c398392519d48cbbbc8aea1c8d2dbdf, State: Initialized, Role: FOLLOWER
I20260808 10:48:02.368568 24981 consensus_queue.cc:260] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:02.368848 24981 ts_tablet_manager.cc:1428] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:48:02.369115 25430 tablet_service.cc:1467] Processing CreateTablet for tablet 8d77811f63cd400ab09e61d4438af1ca (DEFAULT_TABLE table=test-table [id=cd10382a60064fab8b9d46bf235bd287]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:02.369107 25234 tablet_service.cc:1467] Processing CreateTablet for tablet 8d77811f63cd400ab09e61d4438af1ca (DEFAULT_TABLE table=test-table [id=cd10382a60064fab8b9d46bf235bd287]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:02.370249 25430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d77811f63cd400ab09e61d4438af1ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:02.370275 25234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d77811f63cd400ab09e61d4438af1ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:02.378464 25553 tablet_bootstrap.cc:492] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff: Bootstrap starting.
I20260808 10:48:02.378950 25552 tablet_bootstrap.cc:492] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57: Bootstrap starting.
I20260808 10:48:02.381114 25553 tablet_bootstrap.cc:654] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:02.381186 25552 tablet_bootstrap.cc:654] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:02.381908 25553 log.cc:824] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:02.381909 25552 log.cc:824] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:02.383940 25552 tablet_bootstrap.cc:492] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57: No bootstrap required, opened a new log
I20260808 10:48:02.383947 25553 tablet_bootstrap.cc:492] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff: No bootstrap required, opened a new log
I20260808 10:48:02.384178 25552 ts_tablet_manager.cc:1397] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 10:48:02.384177 25553 ts_tablet_manager.cc:1397] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:48:02.387395 25553 raft_consensus.cc:348] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:02.387396 25552 raft_consensus.cc:348] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:02.387633 25553 raft_consensus.cc:374] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:02.387669 25552 raft_consensus.cc:374] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:02.387668 25553 raft_consensus.cc:729] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 941a0de7d1364e8293277b9514cbf7ff, State: Initialized, Role: FOLLOWER
I20260808 10:48:02.387774 25552 raft_consensus.cc:729] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 527f950bf42e45f9940ed23462ab8a57, State: Initialized, Role: FOLLOWER
I20260808 10:48:02.388218 25553 consensus_queue.cc:260] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:02.388276 25552 consensus_queue.cc:260] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:02.388967 25529 heartbeater.cc:503] Master 127.31.237.190:33319 was elected leader, sending a full tablet report...
I20260808 10:48:02.389309 25552 ts_tablet_manager.cc:1428] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 10:48:02.389940 25332 heartbeater.cc:503] Master 127.31.237.190:33319 was elected leader, sending a full tablet report...
I20260808 10:48:02.391222 25553 ts_tablet_manager.cc:1428] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 10:48:02.482439 25133 raft_consensus.cc:482] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:48:02.482801 25133 raft_consensus.cc:504] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:02.484309 25133 leader_election.cc:288] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991), 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803)
I20260808 10:48:02.489966 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.484972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:48765 (local address 127.31.237.130:33803)
0808 10:48:02.485101 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:02.485106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:02.485123 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:48:02.485285 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:02.485288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:02.485337 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:02.485422 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:02.485848 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.486671 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.488592 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.489255 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.489368 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.489423 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:02.489749 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 10:48:02.489868 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:48:02.489965 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.484730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:59123 (local address 127.31.237.131:41991)
0808 10:48:02.484878 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:02.484885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:02.484978 (+ 93us) server_negotiation.cc:407] Connection header received
0808 10:48:02.485267 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:02.485271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:02.485336 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:02.485421 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:02.485848 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.486671 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.488598 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.489211 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.489368 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.489423 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:02.489749 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 10:48:02.489831 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:48:02.490063 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.484498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:41991 (local address 127.31.237.129:59123)
0808 10:48:02.484877 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:02.484894 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:02.485080 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:02.485474 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:02.485478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:02.485529 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:02.485706 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:02.485722 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.486836 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.486839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:02.488377 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:02.488390 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.489386 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.489390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:02.489559 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.489632 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 10:48:02.489711 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 10:48:02.489857 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":323,"spinlock_wait_cycles":8832,"thread_start_us":103,"threads_started":1}
I20260808 10:48:02.490293 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.484568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:33803 (local address 127.31.237.129:48765)
0808 10:48:02.485065 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:02.485079 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:02.485164 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:02.485474 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:02.485478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:02.485527 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:02.485706 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:02.485714 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.486808 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.486815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:02.488377 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:02.488390 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.489386 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.489390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:02.489547 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.489626 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 10:48:02.489711 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:48:02.489857 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":224,"spinlock_wait_cycles":80512,"thread_start_us":69,"threads_started":1}
I20260808 10:48:02.490932 25458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d77811f63cd400ab09e61d4438af1ca" candidate_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "941a0de7d1364e8293277b9514cbf7ff" is_pre_election: true
I20260808 10:48:02.491088 25261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d77811f63cd400ab09e61d4438af1ca" candidate_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "527f950bf42e45f9940ed23462ab8a57" is_pre_election: true
I20260808 10:48:02.491315 25458 raft_consensus.cc:2454] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c398392519d48cbbbc8aea1c8d2dbdf in term 0.
I20260808 10:48:02.491462 25261 raft_consensus.cc:2454] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c398392519d48cbbbc8aea1c8d2dbdf in term 0.
I20260808 10:48:02.491817 24974 leader_election.cc:302] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c398392519d48cbbbc8aea1c8d2dbdf, 941a0de7d1364e8293277b9514cbf7ff; no voters:
I20260808 10:48:02.492131 25133 raft_consensus.cc:2781] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:48:02.492224 25133 raft_consensus.cc:482] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:48:02.492278 25133 raft_consensus.cc:3037] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:02.493340 25133 raft_consensus.cc:504] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:02.493696 25133 leader_election.cc:288] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 1 election: Requested vote from peers 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991), 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803)
I20260808 10:48:02.494016 25458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d77811f63cd400ab09e61d4438af1ca" candidate_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "941a0de7d1364e8293277b9514cbf7ff"
I20260808 10:48:02.494098 25261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d77811f63cd400ab09e61d4438af1ca" candidate_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "527f950bf42e45f9940ed23462ab8a57"
I20260808 10:48:02.494148 25458 raft_consensus.cc:3037] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:02.494237 25261 raft_consensus.cc:3037] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:02.495609 25458 raft_consensus.cc:2454] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c398392519d48cbbbc8aea1c8d2dbdf in term 1.
I20260808 10:48:02.495609 25261 raft_consensus.cc:2454] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c398392519d48cbbbc8aea1c8d2dbdf in term 1.
I20260808 10:48:02.495918 24974 leader_election.cc:302] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c398392519d48cbbbc8aea1c8d2dbdf, 941a0de7d1364e8293277b9514cbf7ff; no voters:
I20260808 10:48:02.496114 25133 raft_consensus.cc:2781] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:02.496304 25133 raft_consensus.cc:686] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 LEADER]: Becoming Leader. State: Replica: 1c398392519d48cbbbc8aea1c8d2dbdf, State: Running, Role: LEADER
I20260808 10:48:02.496569 25133 consensus_queue.cc:237] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:02.499614 24850 catalog_manager.cc:5495] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c398392519d48cbbbc8aea1c8d2dbdf (127.31.237.129). New cstate: current_term: 1 leader_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } health_report { overall_health: HEALTHY } } }
I20260808 10:48:02.523862 25570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.519651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54034 (local address 127.31.237.129:43819)
0808 10:48:02.519986 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:02.519995 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:02.520007 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:48:02.520081 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:02.520083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:02.520140 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:02.520317 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:48:02.520606 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.521180 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.522133 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:02.522394 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:02.522525 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:02.522932 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:48:02.523391 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:48:02.523542 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:02.523612 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 10:48:02.523669 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":99,"threads_started":1}
I20260808 10:48:02.528321 25458 raft_consensus.cc:1261] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff [term 1 FOLLOWER]: Refusing update from remote peer 1c398392519d48cbbbc8aea1c8d2dbdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:48:02.528329 25261 raft_consensus.cc:1261] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57 [term 1 FOLLOWER]: Refusing update from remote peer 1c398392519d48cbbbc8aea1c8d2dbdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:48:02.529177 25562 consensus_queue.cc:1035] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:48:02.529399 25133 consensus_queue.cc:1035] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:48:02.537940 25572 mvcc.cc:205] Tried to move back new op lower bound from 7316218194027765760 to 7316218193908523008. Current Snapshot: MvccSnapshot[applied={T|T < 7316218194027765760}]
I20260808 10:48:02.547631 24852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:48:02.547868 24852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:48:02.550486 24852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Sending DeleteTablet for 3 replicas of tablet 8d77811f63cd400ab09e61d4438af1ca
I20260808 10:48:02.551255 25430 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d77811f63cd400ab09e61d4438af1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:02 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:47182
I20260808 10:48:02.551314 25234 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d77811f63cd400ab09e61d4438af1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:02 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:50930
I20260808 10:48:02.551862 25587 tablet_replica.cc:330] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff: stopping tablet replica
I20260808 10:48:02.551890 25034 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d77811f63cd400ab09e61d4438af1ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:02 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:49910
I20260808 10:48:02.551977 25588 tablet_replica.cc:330] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57: stopping tablet replica
I20260808 10:48:02.552285 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 24921
I20260808 10:48:02.552294 25588 raft_consensus.cc:2229] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:48:02.552683 25588 raft_consensus.cc:2258] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:02.552295 25587 raft_consensus.cc:2229] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:48:02.560276 25587 raft_consensus.cc:2258] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:02.561803 25588 ts_tablet_manager.cc:1905] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:02.561870 25587 ts_tablet_manager.cc:1905] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:02.564221 25588 ts_tablet_manager.cc:1918] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:02.564369 25588 log.cc:1196] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/8d77811f63cd400ab09e61d4438af1ca
I20260808 10:48:02.564705 25588 ts_tablet_manager.cc:1939] T 8d77811f63cd400ab09e61d4438af1ca P 527f950bf42e45f9940ed23462ab8a57: Deleting consensus metadata
I20260808 10:48:02.565783 25587 ts_tablet_manager.cc:1918] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:02.565913 25587 log.cc:1196] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/8d77811f63cd400ab09e61d4438af1ca
I20260808 10:48:02.566272 24828 catalog_manager.cc:4841] TS 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803): tablet 8d77811f63cd400ab09e61d4438af1ca (table test-table [id=cd10382a60064fab8b9d46bf235bd287]) successfully deleted
I20260808 10:48:02.566329 25587 ts_tablet_manager.cc:1939] T 8d77811f63cd400ab09e61d4438af1ca P 941a0de7d1364e8293277b9514cbf7ff: Deleting consensus metadata
I20260808 10:48:02.567914 24830 catalog_manager.cc:4841] TS 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991): tablet 8d77811f63cd400ab09e61d4438af1ca (table test-table [id=cd10382a60064fab8b9d46bf235bd287]) successfully deleted
W20260808 10:48:02.569015 24833 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:43819 (error 108)
I20260808 10:48:02.569298 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 25130
I20260808 10:48:02.570469 25542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:02.570075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:43819 (local address 127.0.0.1:54040)
0808 10:48:02.570243 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:02.570361 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260808 10:48:02.572309 24833 catalog_manager.cc:4584] TS 1c398392519d48cbbbc8aea1c8d2dbdf (127.31.237.129:43819): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d77811f63cd400ab09e61d4438af1ca: Network error: Client connection negotiation failed: client connection to 127.31.237.129:43819: connect: Connection refused (error 111)
I20260808 10:48:02.580107 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 25341
I20260808 10:48:02.599081 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 24806
I20260808 10:48:22.612751 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:33319
--webserver_interface=127.31.237.190
--webserver_port=44463
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:33319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:22.731815 25594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:22.732116 25594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:22.732196 25594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:22.736351 25594 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:48:22.736423 25594 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:48:22.736449 25594 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:48:22.736474 25594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:48:22.736515 25594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:22.736554 25594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:22.736591 25594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:48:22.736629 25594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:48:22.742053 25594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:33319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:33319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=44463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:22.743235 25594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:22.744706 25594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:22.754163 25602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:22.754271 25594 server_base.cc:1034] running on GCE node
W20260808 10:48:22.754169 25604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:22.754181 25601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:22.755091 25594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:22.756225 25594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:22.757447 25594 hybrid_clock.cc:648] HybridClock initialized: now 1786186102757427 us; error 31 us; skew 500 ppm
Aug 08 10:48:22 dist-test-slave-twwt krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:22.760618 25594 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:48:22.761608 25594 webserver.cc:460] Webserver started at http://127.31.237.190:44463/ using document root <none> and password file <none>
I20260808 10:48:22.762209 25594 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:22.762311 25594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:22.766449 25594 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:48:22.768854 25615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:22.770251 25594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:48:22.770438 25594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "0245337fc0694a92ae345f8d88be22aa"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "df375d2c88868456420f9954a8221257"
server_key_iv: "cd39205438d6e65dd3e5a1659140623c"
server_key_version: "encryptionkey@0"
I20260808 10:48:22.771131 25594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:22.785166 25594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:22.788470 25594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:22.788692 25594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:22.796288 25594 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:33319
I20260808 10:48:22.796602 25709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:33319 every 8 connection(s)
I20260808 10:48:22.797626 25594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:48:22.801183 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 25594
I20260808 10:48:22.801724 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:43819
--local_ip_for_outbound_sockets=127.31.237.129
--tserver_master_addrs=127.31.237.190:33319
--webserver_port=35217
--webserver_interface=127.31.237.129
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:48:22.802667 25710 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:48:22.804948 25710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Bootstrap starting.
I20260808 10:48:22.823496 25710 log.cc:824] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:22.836329 25710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:48:22.836797 25710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Bootstrap complete.
I20260808 10:48:22.841166 25710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } }
I20260808 10:48:22.841579 25710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0245337fc0694a92ae345f8d88be22aa, State: Initialized, Role: FOLLOWER
I20260808 10:48:22.842096 25710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } }
I20260808 10:48:22.842214 25710 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:48:22.842304 25710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:48:22.842427 25710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 2 FOLLOWER]: Advancing to term 3
I20260808 10:48:22.844405 25710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } }
I20260808 10:48:22.844753 25710 leader_election.cc:302] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0245337fc0694a92ae345f8d88be22aa; no voters:
I20260808 10:48:22.845137 25710 leader_election.cc:288] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 10:48:22.845223 25716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 3 FOLLOWER]: Leader election won for term 3
I20260808 10:48:22.845448 25716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [term 3 LEADER]: Becoming Leader. State: Replica: 0245337fc0694a92ae345f8d88be22aa, State: Running, Role: LEADER
I20260808 10:48:22.845750 25716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } }
I20260808 10:48:22.846437 25710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:48:22.846741 25716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0245337fc0694a92ae345f8d88be22aa. Latest consensus state: current_term: 3 leader_uuid: "0245337fc0694a92ae345f8d88be22aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } } }
I20260808 10:48:22.846868 25716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:22.847291 25717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0245337fc0694a92ae345f8d88be22aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0245337fc0694a92ae345f8d88be22aa" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33319 } } }
I20260808 10:48:22.847498 25717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:22.848701 25723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:48:22.852949 25723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:48:22.853505 25723 catalog_manager.cc:1259] Loaded cluster ID: 24e3c81fd1c944f099581943be50e9b8
I20260808 10:48:22.853585 25723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:48:22.855427 25723 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:48:22.856050 25723 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Loaded TSK: 0
I20260808 10:48:22.856139 25723 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0245337fc0694a92ae345f8d88be22aa: Loaded TSK: 1
F20260808 10:48:22.856736 25723 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f14d1d3f9ad google::LogMessage::Fail() at ??:0
@ 0x7f14d1d4288a google::LogMessage::SendToLog() at ??:0
@ 0x7f14d1d3f4de google::LogMessage::Flush() at ??:0
@ 0x7f14d1d40a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f14d4a0318d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f14d49fbf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f14dc6a87bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f14dc68299c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f14dc683178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f14dc6c4df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f14dc6e382b std::function<>::operator()() at ??:0
@ 0x7f14dc682b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f14dc683f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f14dc680579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f14dc6c4640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f14d3f913d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f14d3f91c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f14d3f933f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f14d3f82512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f14d55c66db start_thread at ??:0
@ 0x7f14d083d61f clone at ??:0
W20260808 10:48:22.948480 25712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:22.948782 25712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:22.948833 25712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:22.953083 25712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:22.953168 25712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:22.953284 25712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:48:22.958734 25712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:43819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=35217
--tserver_master_addrs=127.31.237.190:33319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:22.959843 25712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:22.960899 25712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:22.971010 25797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:22.971079 25790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:22.971195 25712 server_base.cc:1034] running on GCE node
W20260808 10:48:22.971010 25791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:22.971707 25712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:22.972436 25712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:22.973601 25712 hybrid_clock.cc:648] HybridClock initialized: now 1786186102973565 us; error 50 us; skew 500 ppm
Aug 08 10:48:22 dist-test-slave-twwt krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:22.976420 25712 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:48:22.977362 25712 webserver.cc:460] Webserver started at http://127.31.237.129:35217/ using document root <none> and password file <none>
I20260808 10:48:22.977948 25712 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:22.978012 25712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:22.982167 25712 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:48:22.984619 25806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:22.986017 25712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:48:22.986157 25712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "1c398392519d48cbbbc8aea1c8d2dbdf"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "57621fe0eaa74412bfc5b1a288c31e9a"
server_key_iv: "27ade598afe73dcca52fecc9c0e70547"
server_key_version: "encryptionkey@0"
I20260808 10:48:22.986786 25712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:23.019973 25712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:23.023687 25712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:23.023926 25712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:23.024609 25712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:23.025861 25817 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:48:23.031746 25712 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:48:23.031850 25712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 10:48:23.031967 25712 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:48:23.033819 25712 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:48:23.033898 25712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 10:48:23.034231 25817 tablet_bootstrap.cc:492] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf: Bootstrap starting.
I20260808 10:48:23.045688 25712 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:43819
I20260808 10:48:23.045959 25977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:43819 every 8 connection(s)
I20260808 10:48:23.046912 25712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:48:23.050448 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 25712
I20260808 10:48:23.051047 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:33803
--local_ip_for_outbound_sockets=127.31.237.130
--tserver_master_addrs=127.31.237.190:33319
--webserver_port=38005
--webserver_interface=127.31.237.130
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:48:23.057286 25817 log.cc:824] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:23.067600 25817 tablet_bootstrap.cc:492] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:48:23.068055 25817 tablet_bootstrap.cc:492] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf: Bootstrap complete.
I20260808 10:48:23.069681 25817 ts_tablet_manager.cc:1397] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf: Time spent bootstrapping tablet: real 0.036s user 0.025s sys 0.007s
I20260808 10:48:23.073011 25817 raft_consensus.cc:348] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:23.073697 25817 raft_consensus.cc:729] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c398392519d48cbbbc8aea1c8d2dbdf, State: Initialized, Role: FOLLOWER
I20260808 10:48:23.074252 25817 consensus_queue.cc:260] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:23.075356 25817 ts_tablet_manager.cc:1428] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260808 10:48:23.110569 25981 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:48:23.048966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:40533)
0808 10:48:23.049491 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:23.049549 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:23.050694 (+ 1145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:23.109600 (+ 58906us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":166,"threads_started":1}
W20260808 10:48:23.111287 25978 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:33319 (0 consecutive failures): Network error: Failed to ping master at 127.31.237.190:33319: Client connection negotiation failed: client connection to 127.31.237.190:33319: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:48:23.112098 25981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:23.111801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:57253)
0808 10:48:23.111941 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:23.112023 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 10:48:23.112895 25981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:23.112677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:54947)
0808 10:48:23.112796 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:23.112842 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 10:48:23.113116 25978 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:48:23.183128 25987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:23.183413 25987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:23.183458 25987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:23.187525 25987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:23.187585 25987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:23.187673 25987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:48:23.193149 25987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:33803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=38005
--tserver_master_addrs=127.31.237.190:33319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:23.194224 25987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:23.195359 25987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:23.205209 26007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:23.205245 26003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:23.205209 26004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:23.205628 25987 server_base.cc:1034] running on GCE node
I20260808 10:48:23.206056 25987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:23.206717 25987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:23.207912 25987 hybrid_clock.cc:648] HybridClock initialized: now 1786186103207903 us; error 34 us; skew 500 ppm
Aug 08 10:48:23 dist-test-slave-twwt krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186103, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:23.210922 25987 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:48:23.211782 25987 webserver.cc:460] Webserver started at http://127.31.237.130:38005/ using document root <none> and password file <none>
I20260808 10:48:23.212312 25987 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:23.212374 25987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:23.216305 25987 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 10:48:23.218427 26015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:23.220117 25987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:48:23.220247 25987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "527f950bf42e45f9940ed23462ab8a57"
format_stamp: "Formatted at 2026-08-08 10:47:40 on dist-test-slave-twwt"
server_key: "f07de554a51a5b47abdea857f2d128c4"
server_key_iv: "44bd9bcf7c55b93dce3c633ebff7cd87"
server_key_version: "encryptionkey@0"
I20260808 10:48:23.220835 25987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:23.245833 25987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:23.249028 25987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:23.249243 25987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:23.249867 25987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:23.250739 25987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:23.250787 25987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:23.250855 25987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:23.250901 25987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:23.260366 25987 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:33803
I20260808 10:48:23.260455 26185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:33803 every 8 connection(s)
I20260808 10:48:23.261425 25987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:48:23.265426 26189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:23.263566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:57511)
0808 10:48:23.264054 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:23.264167 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":130,"threads_started":1}
W20260808 10:48:23.265972 26186 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:33319 (0 consecutive failures): Network error: Failed to ping master at 127.31.237.190:33319: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
I20260808 10:48:23.266592 26189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:23.266369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:55653)
0808 10:48:23.266488 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:23.266537 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 10:48:23.267227 26189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:23.267041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:35787)
0808 10:48:23.267136 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:23.267178 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 10:48:23.267413 26186 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:48:23.270915 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 25987
I20260808 10:48:23.271346 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:41991
--local_ip_for_outbound_sockets=127.31.237.131
--tserver_master_addrs=127.31.237.190:33319
--webserver_port=44045
--webserver_interface=127.31.237.131
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:23.391238 26190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:23.391551 26190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:23.391626 26190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:23.395823 26190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:23.395925 26190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:23.396024 26190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:48:23.401525 26190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:41991
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=44045
--tserver_master_addrs=127.31.237.190:33319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:23.402873 26190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:23.404038 26190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:23.414395 26201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:23.414433 26198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:23.414400 26199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:23.414683 26190 server_base.cc:1034] running on GCE node
I20260808 10:48:23.415238 26190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:23.415912 26190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:23.417133 26190 hybrid_clock.cc:648] HybridClock initialized: now 1786186103417093 us; error 52 us; skew 500 ppm
Aug 08 10:48:23 dist-test-slave-twwt krb5kdc[23967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186103, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:23.420163 26190 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:48:23.421096 26190 webserver.cc:460] Webserver started at http://127.31.237.131:44045/ using document root <none> and password file <none>
I20260808 10:48:23.421712 26190 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:23.421810 26190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:23.425997 26190 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:48:23.428140 26214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:23.429446 26190 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 10:48:23.429592 26190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "941a0de7d1364e8293277b9514cbf7ff"
format_stamp: "Formatted at 2026-08-08 10:47:41 on dist-test-slave-twwt"
server_key: "15f91363b32b97db4b129c6cc00b54bd"
server_key_iv: "c6a7cb7ac6cd22f87fbbc3b57a919d15"
server_key_version: "encryptionkey@0"
I20260808 10:48:23.430166 26190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:23.444134 26190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:23.447233 26190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:23.447436 26190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:23.448045 26190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:23.448956 26190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:23.449024 26190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:23.449091 26190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:23.449141 26190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:23.458690 26190 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:41991
I20260808 10:48:23.458705 26420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:41991 every 8 connection(s)
I20260808 10:48:23.459775 26190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:48:23.463753 26424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:23.461867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:57445)
0808 10:48:23.462385 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:23.462498 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":129,"threads_started":1}
W20260808 10:48:23.464248 26422 heartbeater.cc:644] Failed to heartbeat to 127.31.237.190:33319 (0 consecutive failures): Network error: Failed to ping master at 127.31.237.190:33319: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
I20260808 10:48:23.464862 26424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:23.464656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:59085)
0808 10:48:23.464758 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:23.464807 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 10:48:23.465516 26424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:23.465301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:43591)
0808 10:48:23.465418 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:23.465464 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 10:48:23.465762 26422 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:48:23.469293 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 26190
I20260808 10:48:24.114542 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:24.113909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:51721)
0808 10:48:24.114265 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:24.114362 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":105,"threads_started":1}
I20260808 10:48:24.268368 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:24.267983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:57329)
0808 10:48:24.268217 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:24.268269 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":71,"threads_started":1}
I20260808 10:48:24.466802 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:24.466328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:51819)
0808 10:48:24.466592 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:24.466696 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":79,"threads_started":1}
I20260808 10:48:24.829293 26434 raft_consensus.cc:482] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:48:24.829502 26434 raft_consensus.cc:504] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:24.830749 26434 leader_election.cc:288] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991), 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803)
Aug 08 10:48:24 dist-test-slave-twwt krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.131@KRBTEST.COM
Aug 08 10:48:24 dist-test-slave-twwt krb5kdc[23967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.130@KRBTEST.COM
I20260808 10:48:24.847767 26437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:24.831051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:33803 (local address 127.31.237.129:58365)
0808 10:48:24.831383 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:24.831399 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:24.831507 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:24.833573 (+ 2066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:24.833579 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:24.834218 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:24.834922 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:24.834932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:24.836845 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:24.836848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:24.838504 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:24.838519 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:24.839757 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:24.839760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:24.839820 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:24.840251 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:24.840262 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:24.842295 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:24.844612 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:24.844619 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:24.844635 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:24.845170 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:24.845577 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:24.845580 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:24.845582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:24.845811 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:24.846588 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:24.846595 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:24.847007 (+ 412us) client_negotiation.cc:769] Sending connection context
0808 10:48:24.847222 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 10:48:24.847561 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":279,"spinlock_wait_cycles":12672,"thread_start_us":65,"threads_started":1}
I20260808 10:48:24.847882 26435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:24.830945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:41991 (local address 127.31.237.129:35203)
0808 10:48:24.831271 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:24.831287 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:24.831394 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:24.833572 (+ 2178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:24.833579 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:24.834218 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:24.834905 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:24.834923 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:24.837051 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:24.837055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:24.838523 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:24.838537 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:24.839285 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:24.839288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:24.839469 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:24.840130 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:24.840157 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:24.841991 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:24.844612 (+ 2621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:24.844619 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:24.844635 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:24.845170 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:24.845577 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:24.845580 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:24.845582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:24.845811 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:24.846588 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:24.846595 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:24.847311 (+ 716us) client_negotiation.cc:769] Sending connection context
0808 10:48:24.847585 (+ 274us) client_negotiation.cc:240] Negotiation successful
0808 10:48:24.847705 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":229,"spinlock_wait_cycles":23424,"thread_start_us":70,"threads_started":1}
I20260808 10:48:24.847867 26440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:24.831201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:58365 (local address 127.31.237.130:33803)
0808 10:48:24.831658 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:24.831669 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:24.831703 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:48:24.832574 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:24.832584 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:24.833075 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:24.833431 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:24.835096 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:24.836641 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:24.838672 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:24.839152 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:24.839300 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:24.842529 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:24.842535 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:24.842554 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:24.842596 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:24.844477 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:24.845298 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:24.845305 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:24.845311 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:24.845471 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:24.845918 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:24.845922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:24.845925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:24.846452 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:24.846593 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:24.847372 (+ 779us) server_negotiation.cc:299] Negotiation successful
0808 10:48:24.847678 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":131,"threads_started":1}
I20260808 10:48:24.849372 26439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:24.831134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:35203 (local address 127.31.237.131:41991)
0808 10:48:24.831573 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:24.831604 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:24.831649 (+ 45us) server_negotiation.cc:407] Connection header received
0808 10:48:24.832574 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:24.832583 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:24.833075 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:24.833430 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:24.835088 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:24.836878 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:24.838661 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:24.839151 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:24.839328 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:24.842242 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:24.842247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:24.842262 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:24.842290 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:24.844477 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:24.845298 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:24.845305 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:24.845311 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:24.845469 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:24.845941 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:24.845944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:24.845945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:24.846452 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:24.846593 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:24.847749 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0808 10:48:24.849243 (+ 1494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":83,"threads_started":1}
W20260808 10:48:24.850710 25807 leader_election.cc:341] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803): Not found: Tablet not found: 8d77811f63cd400ab09e61d4438af1ca
W20260808 10:48:24.851933 25810 leader_election.cc:341] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991): Not found: Tablet not found: 8d77811f63cd400ab09e61d4438af1ca
I20260808 10:48:24.852044 25810 leader_election.cc:302] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c398392519d48cbbbc8aea1c8d2dbdf; no voters: 527f950bf42e45f9940ed23462ab8a57, 941a0de7d1364e8293277b9514cbf7ff
I20260808 10:48:24.852288 26434 raft_consensus.cc:2726] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:48:25.115754 26435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:25.115422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:59159)
0808 10:48:25.115576 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:25.115665 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 10:48:25.269719 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:25.269261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:34727)
0808 10:48:25.269538 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:25.269600 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":94,"threads_started":1}
I20260808 10:48:25.467981 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:25.467595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:47007)
0808 10:48:25.467840 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:25.467896 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":73,"threads_started":1}
I20260808 10:48:26.117038 26443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:26.116564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:45645)
0808 10:48:26.116868 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:26.116934 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":83,"threads_started":1}
I20260808 10:48:26.271102 26444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:26.270486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:58247)
0808 10:48:26.270931 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:26.271003 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":160,"threads_started":1}
I20260808 10:48:26.469131 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:26.468738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:34697)
0808 10:48:26.468986 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:26.469040 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":79,"threads_started":1}
I20260808 10:48:26.483059 26448 raft_consensus.cc:482] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:48:26.483196 26448 raft_consensus.cc:504] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:26.483632 26448 leader_election.cc:288] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991), 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803)
W20260808 10:48:26.484496 25807 leader_election.cc:341] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803): Not found: Tablet not found: 8d77811f63cd400ab09e61d4438af1ca
W20260808 10:48:26.484602 25810 leader_election.cc:341] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991): Not found: Tablet not found: 8d77811f63cd400ab09e61d4438af1ca
I20260808 10:48:26.484720 25810 leader_election.cc:302] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c398392519d48cbbbc8aea1c8d2dbdf; no voters: 527f950bf42e45f9940ed23462ab8a57, 941a0de7d1364e8293277b9514cbf7ff
I20260808 10:48:26.484935 26448 raft_consensus.cc:2726] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:48:27.118257 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:27.117825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:53955)
0808 10:48:27.118103 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:27.118158 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":85,"threads_started":1}
I20260808 10:48:27.272332 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:27.271876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:57895)
0808 10:48:27.272155 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:27.272213 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":100,"threads_started":1}
I20260808 10:48:27.470409 26453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:27.469912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:55945)
0808 10:48:27.470200 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:27.470285 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":105,"threads_started":1}
I20260808 10:48:28.119818 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:28.119348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:37609)
0808 10:48:28.119656 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:28.119718 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":81,"threads_started":1}
I20260808 10:48:28.173070 26456 raft_consensus.cc:482] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:48:28.173247 26456 raft_consensus.cc:504] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:28.173736 26456 leader_election.cc:288] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991), 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803)
W20260808 10:48:28.174585 25810 leader_election.cc:341] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991): Not found: Tablet not found: 8d77811f63cd400ab09e61d4438af1ca
W20260808 10:48:28.174773 25807 leader_election.cc:341] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803): Not found: Tablet not found: 8d77811f63cd400ab09e61d4438af1ca
I20260808 10:48:28.174935 25807 leader_election.cc:302] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c398392519d48cbbbc8aea1c8d2dbdf; no voters: 527f950bf42e45f9940ed23462ab8a57, 941a0de7d1364e8293277b9514cbf7ff
I20260808 10:48:28.175112 26456 raft_consensus.cc:2726] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:48:28.273579 26457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:28.273158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:45477)
0808 10:48:28.273425 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:28.273484 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":79,"threads_started":1}
I20260808 10:48:28.471763 26460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:28.471295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:50611)
0808 10:48:28.471595 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:28.471661 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":84,"threads_started":1}
I20260808 10:48:29.121218 26463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:29.120709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:54019)
0808 10:48:29.121029 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:29.121091 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":111,"threads_started":1}
I20260808 10:48:29.274915 26464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:29.274400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:46587)
0808 10:48:29.274723 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:29.274784 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":121,"threads_started":1}
I20260808 10:48:29.473133 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:29.472698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:43729)
0808 10:48:29.472983 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:29.473038 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":90,"threads_started":1}
I20260808 10:48:30.122565 26468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:30.122122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:40387)
0808 10:48:30.122398 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:30.122462 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":98,"threads_started":1}
I20260808 10:48:30.276229 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:30.275694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:56979)
0808 10:48:30.276075 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:30.276134 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":81,"threads_started":1}
I20260808 10:48:30.474442 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:30.473886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:40157)
0808 10:48:30.474272 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:30.474339 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":114,"threads_started":1}
I20260808 10:48:31.048017 26471 raft_consensus.cc:482] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:48:31.048228 26471 raft_consensus.cc:504] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:31.048717 26471 leader_election.cc:288] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991), 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803)
W20260808 10:48:31.049445 25810 leader_election.cc:341] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991): Not found: Tablet not found: 8d77811f63cd400ab09e61d4438af1ca
W20260808 10:48:31.049544 25807 leader_election.cc:341] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803): Not found: Tablet not found: 8d77811f63cd400ab09e61d4438af1ca
I20260808 10:48:31.049657 25807 leader_election.cc:302] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c398392519d48cbbbc8aea1c8d2dbdf; no voters: 527f950bf42e45f9940ed23462ab8a57, 941a0de7d1364e8293277b9514cbf7ff
I20260808 10:48:31.049798 26471 raft_consensus.cc:2726] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:48:31.124012 26473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:31.123454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:51321)
0808 10:48:31.123838 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:31.123907 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":89,"threads_started":1}
I20260808 10:48:31.277486 26474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:31.277012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:33869)
0808 10:48:31.277320 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:31.277385 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":98,"threads_started":1}
I20260808 10:48:31.475795 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:31.475338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:43299)
0808 10:48:31.475619 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:31.475686 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":98,"threads_started":1}
I20260808 10:48:32.125645 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:32.124972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:47883)
0808 10:48:32.125411 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:32.125506 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":132,"threads_started":1}
I20260808 10:48:32.279024 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:32.278515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:58711)
0808 10:48:32.278846 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:32.278923 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":158,"threads_started":1}
I20260808 10:48:32.477015 26484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:32.476589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:35291)
0808 10:48:32.476852 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:32.476915 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
I20260808 10:48:33.127008 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:33.126534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:44057)
0808 10:48:33.126843 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:33.126903 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":131,"threads_started":1}
I20260808 10:48:33.280359 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:33.279940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:58185)
0808 10:48:33.280193 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:33.280253 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":79,"threads_started":1}
I20260808 10:48:33.478346 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:33.477880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:49087)
0808 10:48:33.478165 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:33.478242 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":82,"threads_started":1}
I20260808 10:48:34.128232 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:34.127813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:34733)
0808 10:48:34.128073 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:34.128132 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":86,"threads_started":1}
I20260808 10:48:34.281682 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:34.281225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:43559)
0808 10:48:34.281514 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:34.281577 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":87,"threads_started":1}
I20260808 10:48:34.479504 26492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:34.479098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:51283)
0808 10:48:34.479351 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:34.479409 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":81,"threads_started":1}
I20260808 10:48:35.129557 26493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:35.129092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:43757)
0808 10:48:35.129387 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:35.129450 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":81,"threads_started":1}
I20260808 10:48:35.282964 26494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:35.282496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:33905)
0808 10:48:35.282795 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:35.282863 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":129,"threads_started":1}
I20260808 10:48:35.342312 26495 raft_consensus.cc:482] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:48:35.342461 26495 raft_consensus.cc:504] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "941a0de7d1364e8293277b9514cbf7ff" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 41991 } } peers { permanent_uuid: "527f950bf42e45f9940ed23462ab8a57" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33803 } } peers { permanent_uuid: "1c398392519d48cbbbc8aea1c8d2dbdf" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 43819 } }
I20260808 10:48:35.342957 26495 leader_election.cc:288] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991), 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803)
W20260808 10:48:35.343773 25810 leader_election.cc:341] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 941a0de7d1364e8293277b9514cbf7ff (127.31.237.131:41991): Not found: Tablet not found: 8d77811f63cd400ab09e61d4438af1ca
W20260808 10:48:35.343899 25807 leader_election.cc:341] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 527f950bf42e45f9940ed23462ab8a57 (127.31.237.130:33803): Not found: Tablet not found: 8d77811f63cd400ab09e61d4438af1ca
I20260808 10:48:35.343995 25807 leader_election.cc:302] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c398392519d48cbbbc8aea1c8d2dbdf; no voters: 527f950bf42e45f9940ed23462ab8a57, 941a0de7d1364e8293277b9514cbf7ff
I20260808 10:48:35.344189 26495 raft_consensus.cc:2726] T 8d77811f63cd400ab09e61d4438af1ca P 1c398392519d48cbbbc8aea1c8d2dbdf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:48:35.480798 26497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:35.480335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:52093)
0808 10:48:35.480600 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:35.480657 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":95,"threads_started":1}
I20260808 10:48:36.131017 26498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:36.130467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:34571)
0808 10:48:36.130811 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:36.130880 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":94,"threads_started":1}
I20260808 10:48:36.284420 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:36.283871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:59623)
0808 10:48:36.284215 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:36.284291 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":95,"threads_started":1}
I20260808 10:48:36.481987 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:36.481533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:46075)
0808 10:48:36.481828 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:36.481886 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":100,"threads_started":1}
I20260808 10:48:37.132329 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:37.131862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:50245)
0808 10:48:37.132140 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:37.132203 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":90,"threads_started":1}
I20260808 10:48:37.285604 26512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:37.285161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:35129)
0808 10:48:37.285444 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:37.285505 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":85,"threads_started":1}
I20260808 10:48:37.483314 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:37.482862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:51815)
0808 10:48:37.483150 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:37.483212 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":105,"threads_started":1}
I20260808 10:48:38.133939 26516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:38.133451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.129:36417)
0808 10:48:38.133698 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:38.133759 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260808 10:48:38.287134 26517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:38.286669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.130:33727)
0808 10:48:38.286959 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:38.287024 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":73,"threads_started":1}
I20260808 10:48:38.477958 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 25712
I20260808 10:48:38.493054 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:38.492585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33319 (local address 127.31.237.131:37835)
0808 10:48:38.492867 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:38.492952 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.190:33319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":107,"threads_started":1}
I20260808 10:48:38.498761 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 25987
I20260808 10:48:38.507052 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 26190
2026-08-08T10:48:38Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59779 ms)
[----------] 30 tests from SecurityITest (255886 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:48:38 dist-test-slave-twwt krb5kdc[26521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:48:38 dist-test-slave-twwt krb5kdc[26521](info): set up 2 sockets
Aug 08 10:48:38 dist-test-slave-twwt krb5kdc[26521](info): commencing operation
krb5kdc: starting...
W20260808 10:48:39.828284 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:48:39 dist-test-slave-twwt krb5kdc[26521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:48:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:48:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:48:39.965785 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:42969
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:37317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:42969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:40.085143 26539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:40.085474 26539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:40.085578 26539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:40.089681 26539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:48:40.089774 26539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:40.089813 26539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:40.089870 26539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:48:40.089927 26539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:48:40.095458 26539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:37317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:42969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:42969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:40.096611 26539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:40.097712 26539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:40.107288 26551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:40.107275 26546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:40.107270 26545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:40.107311 26539 server_base.cc:1034] running on GCE node
I20260808 10:48:40.108146 26539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:40.109356 26539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:40.110560 26539 hybrid_clock.cc:648] HybridClock initialized: now 1786186120110542 us; error 31 us; skew 500 ppm
Aug 08 10:48:40 dist-test-slave-twwt krb5kdc[26521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186120, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:40.113484 26539 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:48:40.114500 26539 webserver.cc:460] Webserver started at http://127.31.237.190:37287/ using document root <none> and password file <none>
I20260808 10:48:40.115130 26539 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:40.115226 26539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:40.115477 26539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:40.117621 26539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "9083f92022174d449d746070526d5c5a"
format_stamp: "Formatted at 2026-08-08 10:48:40 on dist-test-slave-twwt"
server_key: "bdedba2bcd688baeb64c32c72e7508de"
server_key_iv: "fb2207720d4c2ae458cf0a802fd1df9c"
server_key_version: "encryptionkey@0"
I20260808 10:48:40.118250 26539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "9083f92022174d449d746070526d5c5a"
format_stamp: "Formatted at 2026-08-08 10:48:40 on dist-test-slave-twwt"
server_key: "bdedba2bcd688baeb64c32c72e7508de"
server_key_iv: "fb2207720d4c2ae458cf0a802fd1df9c"
server_key_version: "encryptionkey@0"
I20260808 10:48:40.122277 26539 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:48:40.125011 26560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:40.126317 26539 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:48:40.126470 26539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "9083f92022174d449d746070526d5c5a"
format_stamp: "Formatted at 2026-08-08 10:48:40 on dist-test-slave-twwt"
server_key: "bdedba2bcd688baeb64c32c72e7508de"
server_key_iv: "fb2207720d4c2ae458cf0a802fd1df9c"
server_key_version: "encryptionkey@0"
I20260808 10:48:40.126595 26539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:40.141043 26539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:40.144348 26539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:40.144568 26539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:40.152360 26539 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:42969
I20260808 10:48:40.152383 26646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:42969 every 8 connection(s)
I20260808 10:48:40.153488 26539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:48:40.156646 26649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:40.162528 26649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a: Bootstrap starting.
I20260808 10:48:40.163575 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 26539
I20260808 10:48:40.163712 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:48:40.164031 32694 external_mini_cluster.cc:1428] Setting key 97c79001e742a1849c6618ed045f22f4
I20260808 10:48:40.165508 26649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:40.166479 26649 log.cc:824] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:40.168823 26649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a: No bootstrap required, opened a new log
I20260808 10:48:40.173179 26649 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9083f92022174d449d746070526d5c5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 42969 } }
I20260808 10:48:40.173489 26649 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:40.173552 26649 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9083f92022174d449d746070526d5c5a, State: Initialized, Role: FOLLOWER
Aug 08 10:48:40 dist-test-slave-twwt krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:48:40.174062 26649 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9083f92022174d449d746070526d5c5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 42969 } }
I20260808 10:48:40.174290 26649 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:48:40.174400 26649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:48:40.174544 26649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:40.175796 26649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9083f92022174d449d746070526d5c5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 42969 } }
I20260808 10:48:40.176156 26649 leader_election.cc:302] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9083f92022174d449d746070526d5c5a; no voters:
I20260808 10:48:40.176483 26649 leader_election.cc:288] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:48:40.176599 26656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:40.176859 26656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [term 1 LEADER]: Becoming Leader. State: Replica: 9083f92022174d449d746070526d5c5a, State: Running, Role: LEADER
I20260808 10:48:40.177227 26656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9083f92022174d449d746070526d5c5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 42969 } }
I20260808 10:48:40.177950 26649 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:48:40.179062 26657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9083f92022174d449d746070526d5c5a. Latest consensus state: current_term: 1 leader_uuid: "9083f92022174d449d746070526d5c5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9083f92022174d449d746070526d5c5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 42969 } } }
I20260808 10:48:40.179517 26657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:40.179404 26659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9083f92022174d449d746070526d5c5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9083f92022174d449d746070526d5c5a" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 42969 } } }
I20260808 10:48:40.179874 26659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:40.180482 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:40.165254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44164 (local address 127.31.237.190:42969)
0808 10:48:40.165780 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:40.165817 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:40.165865 (+ 48us) server_negotiation.cc:407] Connection header received
0808 10:48:40.166817 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:40.166846 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:40.167225 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:40.167620 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:40.168301 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.169791 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.170677 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.171404 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.171608 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:40.174461 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:40.174468 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:40.174488 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:40.174526 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:40.176445 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:40.177051 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:40.177057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:40.177063 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:40.177207 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:40.177530 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:40.177533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:40.177535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:40.178114 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:40.178307 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:40.179037 (+ 730us) server_negotiation.cc:299] Negotiation successful
0808 10:48:40.179386 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":152,"threads_started":1}
W20260808 10:48:40.183624 26677 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:48:40.183694 26677 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:48:40.183782 26671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:48:40.184624 26671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:48:40.190941 26671 catalog_manager.cc:1347] Generated new cluster ID: 6e4aa6ee05094097857fab0499a2ad4f
I20260808 10:48:40.191041 26671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:48:40.201681 26671 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:48:40.202826 26671 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:48:40.208559 26671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a: Generated new TSK 0
I20260808 10:48:40.209352 26671 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:48:40.247530 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:42969
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:37317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:40.370328 26686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:40.370599 26686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:40.370766 26686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:40.374914 26686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:40.374979 26686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:40.375075 26686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:48:40.380724 26686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:37317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:42969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:40.381986 26686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:40.383116 26686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:40.393239 26698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:40.393218 26696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:40.393379 26695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:40.393648 26686 server_base.cc:1034] running on GCE node
I20260808 10:48:40.394037 26686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:40.394774 26686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:40.395947 26686 hybrid_clock.cc:648] HybridClock initialized: now 1786186120395911 us; error 50 us; skew 500 ppm
Aug 08 10:48:40 dist-test-slave-twwt krb5kdc[26521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186120, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:40.398877 26686 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:48:40.399806 26686 webserver.cc:460] Webserver started at http://127.31.237.129:34381/ using document root <none> and password file <none>
I20260808 10:48:40.400447 26686 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:40.400511 26686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:40.400739 26686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:40.402951 26686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "b7eb2b002737430b81e0728bd161ddc3"
format_stamp: "Formatted at 2026-08-08 10:48:40 on dist-test-slave-twwt"
server_key: "6d2ce22521a3a08d6cd0a9cf83ef41e4"
server_key_iv: "d14a9fd562df6a66599e9e946f4bdb16"
server_key_version: "encryptionkey@0"
I20260808 10:48:40.403544 26686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "b7eb2b002737430b81e0728bd161ddc3"
format_stamp: "Formatted at 2026-08-08 10:48:40 on dist-test-slave-twwt"
server_key: "6d2ce22521a3a08d6cd0a9cf83ef41e4"
server_key_iv: "d14a9fd562df6a66599e9e946f4bdb16"
server_key_version: "encryptionkey@0"
I20260808 10:48:40.407461 26686 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:48:40.409941 26709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:40.411181 26686 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:48:40.411321 26686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "b7eb2b002737430b81e0728bd161ddc3"
format_stamp: "Formatted at 2026-08-08 10:48:40 on dist-test-slave-twwt"
server_key: "6d2ce22521a3a08d6cd0a9cf83ef41e4"
server_key_iv: "d14a9fd562df6a66599e9e946f4bdb16"
server_key_version: "encryptionkey@0"
I20260808 10:48:40.411438 26686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:40.428488 26686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:40.431813 26686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:40.432073 26686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:40.432775 26686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:40.433743 26686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:40.433789 26686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:40.433840 26686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:40.433876 26686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:40.443341 26686 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:34355
I20260808 10:48:40.443379 26898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:34355 every 8 connection(s)
I20260808 10:48:40.444267 26686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:48:40.445477 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 26686
I20260808 10:48:40.445629 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:48:40.446066 32694 external_mini_cluster.cc:1428] Setting key 4706c80f0b898aa746fa83e5a9c56bce
Aug 08 10:48:40 dist-test-slave-twwt krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186120, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:48:40.459414 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:40.446250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:56253 (local address 127.31.237.190:42969)
0808 10:48:40.446422 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:40.446443 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:40.447484 (+ 1041us) server_negotiation.cc:407] Connection header received
0808 10:48:40.448931 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:40.448937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:40.449000 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:40.449086 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:40.450599 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.451190 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.452390 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.452645 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.452773 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:40.455743 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:40.455746 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:40.455748 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:40.455773 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:40.457214 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:40.457827 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:40.457830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:40.457832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:40.457911 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:40.458207 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:40.458209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:40.458210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:40.458431 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:40.458545 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:40.459096 (+ 551us) server_negotiation.cc:299] Negotiation successful
0808 10:48:40.459227 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
I20260808 10:48:40.460325 26905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:40.446806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:42969 (local address 127.31.237.129:56253)
0808 10:48:40.447329 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:40.447384 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:40.448700 (+ 1316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:40.449222 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:40.449228 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:40.449667 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:40.450426 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:40.450450 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.451315 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.451318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:40.452249 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:40.452260 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.452819 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.452823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:40.453000 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:40.453642 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:40.453658 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:40.455572 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:40.457342 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:40.457351 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:40.457379 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:40.457736 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:40.457994 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:40.457996 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:40.457997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:40.458128 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:40.458546 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:40.458555 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:40.458849 (+ 294us) client_negotiation.cc:769] Sending connection context
0808 10:48:40.459051 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 10:48:40.459389 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":152,"threads_started":1}
I20260808 10:48:40.461681 26899 heartbeater.cc:347] Connected to a master server at 127.31.237.190:42969
I20260808 10:48:40.461987 26899 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:40.462582 26899 heartbeater.cc:511] Master 127.31.237.190:42969 requested a full tablet report, sending...
I20260808 10:48:40.464300 26596 ts_manager.cc:194] Registered new tserver with Master: b7eb2b002737430b81e0728bd161ddc3 (127.31.237.129:34355)
I20260808 10:48:40.465467 26596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:56253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:48:40.478806 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:42969
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:37317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:40.598485 26908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:40.598817 26908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:40.598906 26908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:40.603035 26908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:40.603091 26908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:40.603192 26908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:48:40.608649 26908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:37317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:42969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:40.609748 26908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:40.611159 26908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:40.621194 26916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:40.621194 26919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:40.621208 26917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:40.621861 26908 server_base.cc:1034] running on GCE node
I20260808 10:48:40.622308 26908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:40.623075 26908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:40.624266 26908 hybrid_clock.cc:648] HybridClock initialized: now 1786186120624232 us; error 50 us; skew 500 ppm
Aug 08 10:48:40 dist-test-slave-twwt krb5kdc[26521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186120, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:40.627288 26908 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:48:40.628216 26908 webserver.cc:460] Webserver started at http://127.31.237.130:43737/ using document root <none> and password file <none>
I20260808 10:48:40.628760 26908 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:40.628820 26908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:40.629009 26908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:40.631258 26908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "2cd44532782447dcb5f402336ddb25a1"
format_stamp: "Formatted at 2026-08-08 10:48:40 on dist-test-slave-twwt"
server_key: "a753889f726e0fb928d4824daff9f255"
server_key_iv: "2fdbb869e25ebc7082cb111dcfd9d15c"
server_key_version: "encryptionkey@0"
I20260808 10:48:40.631805 26908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "2cd44532782447dcb5f402336ddb25a1"
format_stamp: "Formatted at 2026-08-08 10:48:40 on dist-test-slave-twwt"
server_key: "a753889f726e0fb928d4824daff9f255"
server_key_iv: "2fdbb869e25ebc7082cb111dcfd9d15c"
server_key_version: "encryptionkey@0"
I20260808 10:48:40.635836 26908 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:48:40.638463 26931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:40.639880 26908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 10:48:40.640017 26908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "2cd44532782447dcb5f402336ddb25a1"
format_stamp: "Formatted at 2026-08-08 10:48:40 on dist-test-slave-twwt"
server_key: "a753889f726e0fb928d4824daff9f255"
server_key_iv: "2fdbb869e25ebc7082cb111dcfd9d15c"
server_key_version: "encryptionkey@0"
I20260808 10:48:40.640136 26908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:40.649323 26908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:40.652218 26908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:40.652420 26908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:40.653038 26908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:40.653873 26908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:40.653929 26908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:40.653998 26908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:40.654033 26908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:40.663290 26908 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:40031
I20260808 10:48:40.663378 27117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:40031 every 8 connection(s)
I20260808 10:48:40.664273 26908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:48:40.666975 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 26908
I20260808 10:48:40.667222 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:48:40.667619 32694 external_mini_cluster.cc:1428] Setting key 8d79a2b55844259302fea86785d3d87f
Aug 08 10:48:40 dist-test-slave-twwt krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186120, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:48:40.680684 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:40.666239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:57709 (local address 127.31.237.190:42969)
0808 10:48:40.666385 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:40.666390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:40.667257 (+ 867us) server_negotiation.cc:407] Connection header received
0808 10:48:40.668829 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:40.668832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:40.668887 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:40.668973 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:40.670192 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.670798 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.672607 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.672840 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.672933 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:40.676207 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:40.676210 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:40.676212 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:40.676241 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:40.677813 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:40.678532 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:40.678551 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:40.678556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:40.678699 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:40.679203 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:40.679208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:40.679211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:40.679522 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:40.679676 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:40.680300 (+ 624us) server_negotiation.cc:299] Negotiation successful
0808 10:48:40.680499 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:48:40.681600 27122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:40.666550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:42969 (local address 127.31.237.130:57709)
0808 10:48:40.667094 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:40.667151 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:40.668598 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:40.669157 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:40.669164 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:40.669534 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:40.670037 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:40.670050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.670929 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.670937 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:40.672468 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:40.672483 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.673016 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.673023 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:40.673270 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:40.673925 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:40.673952 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:40.675994 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:40.677942 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:40.677950 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:40.677968 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:40.678366 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:40.678847 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:40.678852 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:40.678856 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:40.679086 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:40.679676 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:40.679684 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:40.680022 (+ 338us) client_negotiation.cc:769] Sending connection context
0808 10:48:40.680253 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 10:48:40.680636 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":162,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
I20260808 10:48:40.683108 27119 heartbeater.cc:347] Connected to a master server at 127.31.237.190:42969
I20260808 10:48:40.683396 27119 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:40.684077 27119 heartbeater.cc:511] Master 127.31.237.190:42969 requested a full tablet report, sending...
I20260808 10:48:40.685411 26596 ts_manager.cc:194] Registered new tserver with Master: 2cd44532782447dcb5f402336ddb25a1 (127.31.237.130:40031)
I20260808 10:48:40.686000 26596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:57709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:48:40.702157 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:42969
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:37317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:40.820822 27125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:40.821133 27125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:40.821213 27125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:40.825333 27125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:40.825438 27125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:40.825537 27125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:48:40.831229 27125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:37317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:42969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:40.832429 27125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:40.833545 27125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:40.843605 27131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:40.843605 27132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:40.843845 27125 server_base.cc:1034] running on GCE node
W20260808 10:48:40.843647 27134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:40.844333 27125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:40.845031 27125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:40.846196 27125 hybrid_clock.cc:648] HybridClock initialized: now 1786186120846142 us; error 65 us; skew 500 ppm
Aug 08 10:48:40 dist-test-slave-twwt krb5kdc[26521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186120, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:40.849088 27125 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:48:40.849953 27125 webserver.cc:460] Webserver started at http://127.31.237.131:41895/ using document root <none> and password file <none>
I20260808 10:48:40.850495 27125 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:40.850554 27125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:40.850795 27125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:40.852878 27125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "8103fc59a4d34482b083310cb35eff04"
format_stamp: "Formatted at 2026-08-08 10:48:40 on dist-test-slave-twwt"
server_key: "ca1577a0aa0a14fbd64f41a5dd51fa07"
server_key_iv: "85206f186fdbce3944464fe8b155bc68"
server_key_version: "encryptionkey@0"
I20260808 10:48:40.853441 27125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "8103fc59a4d34482b083310cb35eff04"
format_stamp: "Formatted at 2026-08-08 10:48:40 on dist-test-slave-twwt"
server_key: "ca1577a0aa0a14fbd64f41a5dd51fa07"
server_key_iv: "85206f186fdbce3944464fe8b155bc68"
server_key_version: "encryptionkey@0"
I20260808 10:48:40.857311 27125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 10:48:40.859790 27146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:40.861130 27125 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:48:40.861277 27125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "8103fc59a4d34482b083310cb35eff04"
format_stamp: "Formatted at 2026-08-08 10:48:40 on dist-test-slave-twwt"
server_key: "ca1577a0aa0a14fbd64f41a5dd51fa07"
server_key_iv: "85206f186fdbce3944464fe8b155bc68"
server_key_version: "encryptionkey@0"
I20260808 10:48:40.861400 27125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:40.874836 27125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:40.878499 27125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:40.878759 27125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:40.879563 27125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:40.880659 27125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:40.880736 27125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:40.880829 27125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:40.880877 27125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:40.891207 27125 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:35543
I20260808 10:48:40.891232 27312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:35543 every 8 connection(s)
I20260808 10:48:40.892247 27125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:48:40.900113 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 27125
I20260808 10:48:40.900277 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:48:40.900666 32694 external_mini_cluster.cc:1428] Setting key e03f5d8a80203ed1fc656b8ff77bd02d
Aug 08 10:48:40 dist-test-slave-twwt krb5kdc[26521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186120, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:48:40.907339 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:40.894232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:56921 (local address 127.31.237.190:42969)
0808 10:48:40.894438 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:40.894443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:40.895363 (+ 920us) server_negotiation.cc:407] Connection header received
0808 10:48:40.896724 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:40.896726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:40.896775 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:40.896858 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:40.898056 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.898596 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.899772 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.899994 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.900077 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:40.903781 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:40.903785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:40.903788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:40.903822 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:40.905397 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:40.905887 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:40.905890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:40.905891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:40.905967 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:40.906270 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:40.906272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:40.906273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:40.906463 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:40.906542 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:40.907117 (+ 575us) server_negotiation.cc:299] Negotiation successful
0808 10:48:40.907218 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260808 10:48:40.908195 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:40.894679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:42969 (local address 127.31.237.131:56921)
0808 10:48:40.895203 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:40.895250 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:40.896506 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:40.897037 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:40.897045 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:40.897399 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:40.897920 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:40.897935 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.898771 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.898774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:40.899662 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:40.899672 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.900168 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.900174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:40.900405 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:40.901172 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:40.901231 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:40.903575 (+ 2344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:40.905519 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:40.905525 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:40.905537 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:40.905798 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:40.906048 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:40.906050 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:40.906052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:40.906185 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:40.906554 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:40.906560 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:40.906893 (+ 333us) client_negotiation.cc:769] Sending connection context
0808 10:48:40.907062 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 10:48:40.907318 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":127,"threads_started":1}
I20260808 10:48:40.909396 27314 heartbeater.cc:347] Connected to a master server at 127.31.237.190:42969
I20260808 10:48:40.909657 27314 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:40.910117 27314 heartbeater.cc:511] Master 127.31.237.190:42969 requested a full tablet report, sending...
I20260808 10:48:40.911178 26596 ts_manager.cc:194] Registered new tserver with Master: 8103fc59a4d34482b083310cb35eff04 (127.31.237.131:35543)
I20260808 10:48:40.911774 26596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:56921
I20260808 10:48:40.915228 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:48:40.925015 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:40.917540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44168 (local address 127.31.237.190:42969)
0808 10:48:40.917667 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:40.917670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:40.917900 (+ 230us) server_negotiation.cc:407] Connection header received
0808 10:48:40.917996 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:40.917998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:40.918034 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:40.918113 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:40.918950 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.919442 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.920382 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.920600 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.920681 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:40.922123 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:40.922126 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:40.922128 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:40.922150 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:40.923617 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:40.923998 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:40.924001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:40.924002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:40.924074 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:40.924314 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:40.924316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:40.924317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:40.924515 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:40.924615 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:40.924810 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 10:48:40.924900 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:48:40.927829 26596 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:48:40.931030 26596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:48:40.946530 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:40.940013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:34355 (local address 127.0.0.1:57274)
0808 10:48:40.941375 (+ 1362us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:40.941390 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:40.941488 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:40.942105 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:40.942108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:40.942126 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:40.942231 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:40.942238 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.943383 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.943391 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:40.944943 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:40.944950 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.945917 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.945921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:40.946041 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:40.946079 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 10:48:40.946224 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 10:48:40.946348 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1300,"mutex_wait_us":227,"thread_start_us":124,"threads_started":1}
I20260808 10:48:40.946758 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:40.940138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58588 (local address 127.31.237.131:35543)
0808 10:48:40.940884 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:40.940900 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:40.940932 (+ 32us) server_negotiation.cc:407] Connection header received
0808 10:48:40.941024 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:40.941030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:40.941285 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:40.941907 (+ 622us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:40.942057 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.943394 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.945064 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.946036 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.946240 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:40.946389 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:40.946452 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 10:48:40.946556 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":167,"threads_started":1}
I20260808 10:48:40.946980 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:40.940845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:40031 (local address 127.0.0.1:42104)
0808 10:48:40.941661 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:40.941674 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:40.941749 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:40.942313 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:40.942316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:40.942338 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:40.942424 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:40.942430 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.943943 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.943950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:40.945111 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:40.945126 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.946490 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.946495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:40.946589 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:40.946644 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 10:48:40.946736 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 10:48:40.946828 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"thread_start_us":106,"threads_started":1}
I20260808 10:48:40.946563 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:40.939970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:35543 (local address 127.0.0.1:58588)
0808 10:48:40.940400 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:40.940448 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:40.940572 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:40.941504 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:40.941516 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:40.941551 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:40.941713 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:40.941723 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.943540 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.943543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:40.944924 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:40.944936 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.946233 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.946237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:40.946320 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:40.946341 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:48:40.946401 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 10:48:40.946457 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":27,"thread_start_us":97,"threads_started":1}
I20260808 10:48:40.947484 27331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:40.940803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57274 (local address 127.31.237.129:34355)
0808 10:48:40.941101 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:40.941120 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:40.941671 (+ 551us) server_negotiation.cc:407] Connection header received
0808 10:48:40.941767 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:40.941775 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:40.941963 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:40.942119 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:40.942357 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.943224 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.945107 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.945782 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.945936 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:40.946089 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:40.947280 (+ 1191us) server_negotiation.cc:299] Negotiation successful
0808 10:48:40.947374 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":137,"threads_started":1}
I20260808 10:48:40.947568 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:40.941341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42104 (local address 127.31.237.130:40031)
0808 10:48:40.941866 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:40.941880 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:40.941902 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:48:40.941978 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:40.941983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:40.942177 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:40.942345 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:40.942515 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.943766 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.945298 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:40.946195 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:40.946312 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:40.946421 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:40.947286 (+ 865us) server_negotiation.cc:299] Negotiation successful
0808 10:48:40.947400 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":160,"threads_started":1}
I20260808 10:48:40.950420 27214 tablet_service.cc:1467] Processing CreateTablet for tablet 8d293fdac0414977b333704ce3a50ec6 (DEFAULT_TABLE table=test-table [id=0feb28bc896d48de87110ff3f9aa71f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:40.950435 26787 tablet_service.cc:1467] Processing CreateTablet for tablet 8d293fdac0414977b333704ce3a50ec6 (DEFAULT_TABLE table=test-table [id=0feb28bc896d48de87110ff3f9aa71f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:40.950423 27007 tablet_service.cc:1467] Processing CreateTablet for tablet 8d293fdac0414977b333704ce3a50ec6 (DEFAULT_TABLE table=test-table [id=0feb28bc896d48de87110ff3f9aa71f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:40.951563 27214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d293fdac0414977b333704ce3a50ec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:40.951574 27007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d293fdac0414977b333704ce3a50ec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:40.951574 26787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d293fdac0414977b333704ce3a50ec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:40.958657 27335 tablet_bootstrap.cc:492] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04: Bootstrap starting.
I20260808 10:48:40.959496 27336 tablet_bootstrap.cc:492] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3: Bootstrap starting.
I20260808 10:48:40.961304 27335 tablet_bootstrap.cc:654] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:40.961576 27336 tablet_bootstrap.cc:654] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:40.961840 27337 tablet_bootstrap.cc:492] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1: Bootstrap starting.
I20260808 10:48:40.962505 27336 log.cc:824] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:40.962822 27335 log.cc:824] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:40.964164 27337 tablet_bootstrap.cc:654] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:40.964748 27336 tablet_bootstrap.cc:492] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3: No bootstrap required, opened a new log
I20260808 10:48:40.964895 27335 tablet_bootstrap.cc:492] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04: No bootstrap required, opened a new log
I20260808 10:48:40.964973 27336 ts_tablet_manager.cc:1397] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 10:48:40.965087 27335 ts_tablet_manager.cc:1397] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260808 10:48:40.965607 27337 log.cc:824] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:40.967702 27337 tablet_bootstrap.cc:492] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1: No bootstrap required, opened a new log
I20260808 10:48:40.967948 27337 ts_tablet_manager.cc:1397] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:48:40.969383 27335 raft_consensus.cc:348] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8103fc59a4d34482b083310cb35eff04" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35543 } } peers { permanent_uuid: "b7eb2b002737430b81e0728bd161ddc3" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34355 } } peers { permanent_uuid: "2cd44532782447dcb5f402336ddb25a1" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 40031 } }
I20260808 10:48:40.969697 27335 raft_consensus.cc:374] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:40.969749 27335 raft_consensus.cc:729] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8103fc59a4d34482b083310cb35eff04, State: Initialized, Role: FOLLOWER
I20260808 10:48:40.969681 27336 raft_consensus.cc:348] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8103fc59a4d34482b083310cb35eff04" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35543 } } peers { permanent_uuid: "b7eb2b002737430b81e0728bd161ddc3" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34355 } } peers { permanent_uuid: "2cd44532782447dcb5f402336ddb25a1" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 40031 } }
I20260808 10:48:40.969938 27336 raft_consensus.cc:374] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:40.969990 27336 raft_consensus.cc:729] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7eb2b002737430b81e0728bd161ddc3, State: Initialized, Role: FOLLOWER
I20260808 10:48:40.970321 27335 consensus_queue.cc:260] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8103fc59a4d34482b083310cb35eff04" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35543 } } peers { permanent_uuid: "b7eb2b002737430b81e0728bd161ddc3" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34355 } } peers { permanent_uuid: "2cd44532782447dcb5f402336ddb25a1" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 40031 } }
I20260808 10:48:40.970391 27336 consensus_queue.cc:260] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8103fc59a4d34482b083310cb35eff04" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35543 } } peers { permanent_uuid: "b7eb2b002737430b81e0728bd161ddc3" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34355 } } peers { permanent_uuid: "2cd44532782447dcb5f402336ddb25a1" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 40031 } }
I20260808 10:48:40.971490 27335 ts_tablet_manager.cc:1428] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260808 10:48:40.971490 27336 ts_tablet_manager.cc:1428] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260808 10:48:40.971673 27314 heartbeater.cc:503] Master 127.31.237.190:42969 was elected leader, sending a full tablet report...
I20260808 10:48:40.972178 27337 raft_consensus.cc:348] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8103fc59a4d34482b083310cb35eff04" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35543 } } peers { permanent_uuid: "b7eb2b002737430b81e0728bd161ddc3" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34355 } } peers { permanent_uuid: "2cd44532782447dcb5f402336ddb25a1" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 40031 } }
I20260808 10:48:40.972442 27337 raft_consensus.cc:374] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:40.972496 27337 raft_consensus.cc:729] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd44532782447dcb5f402336ddb25a1, State: Initialized, Role: FOLLOWER
I20260808 10:48:40.973091 26899 heartbeater.cc:503] Master 127.31.237.190:42969 was elected leader, sending a full tablet report...
I20260808 10:48:40.973019 27337 consensus_queue.cc:260] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8103fc59a4d34482b083310cb35eff04" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35543 } } peers { permanent_uuid: "b7eb2b002737430b81e0728bd161ddc3" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34355 } } peers { permanent_uuid: "2cd44532782447dcb5f402336ddb25a1" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 40031 } }
I20260808 10:48:40.973765 27119 heartbeater.cc:503] Master 127.31.237.190:42969 was elected leader, sending a full tablet report...
I20260808 10:48:40.974161 27337 ts_tablet_manager.cc:1428] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
W20260808 10:48:41.143416 27316 tablet.cc:2369] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:48:41.165514 27120 tablet.cc:2369] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:48:41.195659 26900 tablet.cc:2369] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:48:41.244658 27345 raft_consensus.cc:482] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:48:41.244932 27345 raft_consensus.cc:504] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8103fc59a4d34482b083310cb35eff04" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35543 } } peers { permanent_uuid: "b7eb2b002737430b81e0728bd161ddc3" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34355 } } peers { permanent_uuid: "2cd44532782447dcb5f402336ddb25a1" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 40031 } }
I20260808 10:48:41.246181 27345 leader_election.cc:288] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8103fc59a4d34482b083310cb35eff04 (127.31.237.131:35543), 2cd44532782447dcb5f402336ddb25a1 (127.31.237.130:40031)
I20260808 10:48:41.250845 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:41.247216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:47523 (local address 127.31.237.130:40031)
0808 10:48:41.247388 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:41.247393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:41.247557 (+ 164us) server_negotiation.cc:407] Connection header received
0808 10:48:41.247656 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:41.247661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:41.247727 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:41.247787 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:41.248080 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.248651 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.249725 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.250122 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.250267 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:41.250335 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:41.250600 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 10:48:41.250706 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 10:48:41.250845 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:41.246682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:40735 (local address 127.31.237.131:35543)
0808 10:48:41.246837 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:41.246841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:41.247002 (+ 161us) server_negotiation.cc:407] Connection header received
0808 10:48:41.247176 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:41.247179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:41.247241 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:41.247342 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:41.247726 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.248390 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.249661 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.250111 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.250266 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:41.250333 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:41.250597 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 10:48:41.250708 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:48:41.250882 27349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:41.247075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:40031 (local address 127.31.237.129:47523)
0808 10:48:41.247451 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:41.247467 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:41.247570 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:41.247826 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:41.247828 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:41.247848 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:41.247964 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:41.247971 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.248768 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.248771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:41.249565 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:41.249576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.250266 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.250270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:41.250430 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:41.250484 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 10:48:41.250580 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 10:48:41.250740 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":86,"thread_start_us":107,"threads_started":1}
I20260808 10:48:41.250895 27348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:41.246522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:35543 (local address 127.31.237.129:40735)
0808 10:48:41.246891 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:41.246909 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:41.247051 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:41.247379 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:41.247384 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:41.247410 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:41.247585 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:41.247594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.248502 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.248506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:41.249518 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:41.249529 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.250267 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.250271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:41.250465 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:41.250485 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 10:48:41.250576 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 10:48:41.250711 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":136,"thread_start_us":181,"threads_started":1}
I20260808 10:48:41.251508 27048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d293fdac0414977b333704ce3a50ec6" candidate_uuid: "b7eb2b002737430b81e0728bd161ddc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd44532782447dcb5f402336ddb25a1" is_pre_election: true
I20260808 10:48:41.251767 27245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d293fdac0414977b333704ce3a50ec6" candidate_uuid: "b7eb2b002737430b81e0728bd161ddc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8103fc59a4d34482b083310cb35eff04" is_pre_election: true
I20260808 10:48:41.251837 27048 raft_consensus.cc:2454] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7eb2b002737430b81e0728bd161ddc3 in term 0.
I20260808 10:48:41.252064 27245 raft_consensus.cc:2454] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7eb2b002737430b81e0728bd161ddc3 in term 0.
I20260808 10:48:41.252349 26716 leader_election.cc:302] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd44532782447dcb5f402336ddb25a1, b7eb2b002737430b81e0728bd161ddc3; no voters:
I20260808 10:48:41.252599 27345 raft_consensus.cc:2781] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:48:41.252683 27345 raft_consensus.cc:482] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:48:41.252723 27345 raft_consensus.cc:3037] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:41.253813 27345 raft_consensus.cc:504] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8103fc59a4d34482b083310cb35eff04" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35543 } } peers { permanent_uuid: "b7eb2b002737430b81e0728bd161ddc3" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34355 } } peers { permanent_uuid: "2cd44532782447dcb5f402336ddb25a1" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 40031 } }
I20260808 10:48:41.254174 27345 leader_election.cc:288] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [CANDIDATE]: Term 1 election: Requested vote from peers 8103fc59a4d34482b083310cb35eff04 (127.31.237.131:35543), 2cd44532782447dcb5f402336ddb25a1 (127.31.237.130:40031)
I20260808 10:48:41.254578 27245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d293fdac0414977b333704ce3a50ec6" candidate_uuid: "b7eb2b002737430b81e0728bd161ddc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8103fc59a4d34482b083310cb35eff04"
I20260808 10:48:41.254669 27048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d293fdac0414977b333704ce3a50ec6" candidate_uuid: "b7eb2b002737430b81e0728bd161ddc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd44532782447dcb5f402336ddb25a1"
I20260808 10:48:41.254765 27245 raft_consensus.cc:3037] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:41.254822 27048 raft_consensus.cc:3037] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:41.255860 27245 raft_consensus.cc:2454] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7eb2b002737430b81e0728bd161ddc3 in term 1.
I20260808 10:48:41.255888 27048 raft_consensus.cc:2454] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7eb2b002737430b81e0728bd161ddc3 in term 1.
I20260808 10:48:41.256175 26716 leader_election.cc:302] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8103fc59a4d34482b083310cb35eff04, b7eb2b002737430b81e0728bd161ddc3; no voters:
I20260808 10:48:41.256369 27345 raft_consensus.cc:2781] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:41.256590 27345 raft_consensus.cc:686] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [term 1 LEADER]: Becoming Leader. State: Replica: b7eb2b002737430b81e0728bd161ddc3, State: Running, Role: LEADER
I20260808 10:48:41.256986 27345 consensus_queue.cc:237] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8103fc59a4d34482b083310cb35eff04" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35543 } } peers { permanent_uuid: "b7eb2b002737430b81e0728bd161ddc3" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34355 } } peers { permanent_uuid: "2cd44532782447dcb5f402336ddb25a1" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 40031 } }
I20260808 10:48:41.260375 26596 catalog_manager.cc:5495] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7eb2b002737430b81e0728bd161ddc3 (127.31.237.129). New cstate: current_term: 1 leader_uuid: "b7eb2b002737430b81e0728bd161ddc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8103fc59a4d34482b083310cb35eff04" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7eb2b002737430b81e0728bd161ddc3" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 34355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cd44532782447dcb5f402336ddb25a1" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260808 10:48:41.313323 27331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:41.309596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57280 (local address 127.31.237.129:34355)
0808 10:48:41.309739 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:41.309742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:41.309771 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:48:41.309883 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:41.309886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:41.309954 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:41.310044 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:48:41.310270 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.310824 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.311818 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.312058 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.312141 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:41.312441 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:48:41.312934 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:48:41.313043 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:41.313150 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 10:48:41.313207 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:48:41.319977 27048 raft_consensus.cc:1261] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1 [term 1 FOLLOWER]: Refusing update from remote peer b7eb2b002737430b81e0728bd161ddc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:48:41.319984 27245 raft_consensus.cc:1261] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04 [term 1 FOLLOWER]: Refusing update from remote peer b7eb2b002737430b81e0728bd161ddc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:48:41.320763 27345 consensus_queue.cc:1035] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8103fc59a4d34482b083310cb35eff04" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:48:41.321081 27350 consensus_queue.cc:1035] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd44532782447dcb5f402336ddb25a1" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:48:41.329346 27357 mvcc.cc:205] Tried to move back new op lower bound from 7316218352918437888 to 7316218352671481856. Current Snapshot: MvccSnapshot[applied={T|T < 7316218352918437888}]
I20260808 10:48:41.329566 27358 mvcc.cc:205] Tried to move back new op lower bound from 7316218352918437888 to 7316218352671481856. Current Snapshot: MvccSnapshot[applied={T|T < 7316218352918437888}]
I20260808 10:48:41.339735 26592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:48:41.339931 26592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:48:41.342496 26592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9083f92022174d449d746070526d5c5a: Sending DeleteTablet for 3 replicas of tablet 8d293fdac0414977b333704ce3a50ec6
I20260808 10:48:41.343379 27214 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d293fdac0414977b333704ce3a50ec6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:41 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:58588
I20260808 10:48:41.343400 26787 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d293fdac0414977b333704ce3a50ec6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:41 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:57274
I20260808 10:48:41.343756 27007 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d293fdac0414977b333704ce3a50ec6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:41 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:42104
I20260808 10:48:41.344257 27374 tablet_replica.cc:330] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3: stopping tablet replica
I20260808 10:48:41.344460 27375 tablet_replica.cc:330] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1: stopping tablet replica
I20260808 10:48:41.344666 27374 raft_consensus.cc:2229] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:48:41.344882 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 26686
I20260808 10:48:41.344878 27375 raft_consensus.cc:2229] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:48:41.345268 27374 raft_consensus.cc:2258] T 8d293fdac0414977b333704ce3a50ec6 P b7eb2b002737430b81e0728bd161ddc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:41.345273 27375 raft_consensus.cc:2258] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:41.347718 27373 tablet_replica.cc:330] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04: stopping tablet replica
I20260808 10:48:41.348042 27373 raft_consensus.cc:2229] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:48:41.348428 27373 raft_consensus.cc:2258] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:41.354732 27373 ts_tablet_manager.cc:1905] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:41.355491 27375 ts_tablet_manager.cc:1905] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:41.357754 27375 ts_tablet_manager.cc:1918] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:41.357885 27375 log.cc:1196] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/8d293fdac0414977b333704ce3a50ec6
I20260808 10:48:41.358193 27375 ts_tablet_manager.cc:1939] T 8d293fdac0414977b333704ce3a50ec6 P 2cd44532782447dcb5f402336ddb25a1: Deleting consensus metadata
I20260808 10:48:41.358402 27373 ts_tablet_manager.cc:1918] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:41.358513 27373 log.cc:1196] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/8d293fdac0414977b333704ce3a50ec6
I20260808 10:48:41.358889 27373 ts_tablet_manager.cc:1939] T 8d293fdac0414977b333704ce3a50ec6 P 8103fc59a4d34482b083310cb35eff04: Deleting consensus metadata
I20260808 10:48:41.359604 26569 catalog_manager.cc:4841] TS 2cd44532782447dcb5f402336ddb25a1 (127.31.237.130:40031): tablet 8d293fdac0414977b333704ce3a50ec6 (table test-table [id=0feb28bc896d48de87110ff3f9aa71f1]) successfully deleted
I20260808 10:48:41.360106 26569 catalog_manager.cc:4841] TS 8103fc59a4d34482b083310cb35eff04 (127.31.237.131:35543): tablet 8d293fdac0414977b333704ce3a50ec6 (table test-table [id=0feb28bc896d48de87110ff3f9aa71f1]) successfully deleted
W20260808 10:48:41.361194 26561 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:34355 (error 108)
I20260808 10:48:41.362082 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 26908
I20260808 10:48:41.362303 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:41.361965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:34355 (local address 127.0.0.1:57290)
0808 10:48:41.362114 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:41.362198 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:34355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 10:48:41.378537 26561 catalog_manager.cc:4584] TS b7eb2b002737430b81e0728bd161ddc3 (127.31.237.129:34355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d293fdac0414977b333704ce3a50ec6: Network error: Client connection negotiation failed: client connection to 127.31.237.129:34355: connect: Connection refused (error 111)
I20260808 10:48:41.388298 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 27125
I20260808 10:48:41.401376 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 26539
2026-08-08T10:48:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2894 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T10:48:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:48:41Z Disabled control of system clock
I20260808 10:48:41.442370 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:45457
--webserver_interface=127.31.237.190
--webserver_port=0
--builtin_ntp_servers=127.31.237.148:46599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:45457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:48:41.561254 27383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:41.561537 27383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:41.561573 27383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:41.565557 27383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:48:41.565623 27383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:41.565639 27383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:41.565656 27383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:48:41.565672 27383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:48:41.570875 27383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:46599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:45457
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:45457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:41.572018 27383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:41.573447 27383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:41.583053 27394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:41.583050 27389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:41.583086 27391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:41.583427 27383 server_base.cc:1034] running on GCE node
I20260808 10:48:41.584087 27383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:41.585379 27383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:41.586580 27383 hybrid_clock.cc:648] HybridClock initialized: now 1786186121586546 us; error 45 us; skew 500 ppm
I20260808 10:48:41.588871 27383 webserver.cc:460] Webserver started at http://127.31.237.190:39723/ using document root <none> and password file <none>
I20260808 10:48:41.589536 27383 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:41.589602 27383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:41.589861 27383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:41.592116 27383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "c1d70d268b1f49138033eb3420004683"
format_stamp: "Formatted at 2026-08-08 10:48:41 on dist-test-slave-twwt"
server_key: "3eea2f4be478c497026085a07573afd8"
server_key_iv: "cd74831be855d7a393045002be221096"
server_key_version: "encryptionkey@0"
I20260808 10:48:41.592711 27383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "c1d70d268b1f49138033eb3420004683"
format_stamp: "Formatted at 2026-08-08 10:48:41 on dist-test-slave-twwt"
server_key: "3eea2f4be478c497026085a07573afd8"
server_key_iv: "cd74831be855d7a393045002be221096"
server_key_version: "encryptionkey@0"
I20260808 10:48:41.596674 27383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:48:41.599407 27402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:41.600792 27383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:48:41.600972 27383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "c1d70d268b1f49138033eb3420004683"
format_stamp: "Formatted at 2026-08-08 10:48:41 on dist-test-slave-twwt"
server_key: "3eea2f4be478c497026085a07573afd8"
server_key_iv: "cd74831be855d7a393045002be221096"
server_key_version: "encryptionkey@0"
I20260808 10:48:41.601102 27383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:41.617280 27383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:41.618052 27383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:41.618279 27383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:41.625638 27383 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:45457
I20260808 10:48:41.625661 27482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:45457 every 8 connection(s)
I20260808 10:48:41.626695 27383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:48:41.629041 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 27383
I20260808 10:48:41.629190 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:48:41.629505 32694 external_mini_cluster.cc:1428] Setting key 14c00561ce52eebd284aaf8a5f5985f2
I20260808 10:48:41.630278 27483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:41.637844 27483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683: Bootstrap starting.
I20260808 10:48:41.639599 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:41.631107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45854 (local address 127.31.237.190:45457)
0808 10:48:41.631796 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:41.631814 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:41.631854 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:48:41.632535 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:41.632553 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:41.632979 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:41.633283 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:41.633703 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.634801 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.636019 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.636510 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.636626 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:41.637601 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:41.637607 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:41.637620 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:41.638095 (+ 475us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:41.638156 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:41.638179 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:41.638257 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:41.638474 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 10:48:41.638831 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":163,"threads_started":1}
I20260808 10:48:41.640476 27483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:41.641412 27483 log.cc:824] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:41.643697 27483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683: No bootstrap required, opened a new log
I20260808 10:48:41.647408 27483 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d70d268b1f49138033eb3420004683" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45457 } }
I20260808 10:48:41.647639 27483 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:41.647684 27483 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d70d268b1f49138033eb3420004683, State: Initialized, Role: FOLLOWER
I20260808 10:48:41.648128 27483 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d70d268b1f49138033eb3420004683" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45457 } }
I20260808 10:48:41.648255 27483 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:48:41.648304 27483 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:48:41.648378 27483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:41.649286 27483 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d70d268b1f49138033eb3420004683" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45457 } }
I20260808 10:48:41.649598 27483 leader_election.cc:302] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1d70d268b1f49138033eb3420004683; no voters:
I20260808 10:48:41.649883 27483 leader_election.cc:288] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:48:41.650044 27490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:41.650305 27490 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [term 1 LEADER]: Becoming Leader. State: Replica: c1d70d268b1f49138033eb3420004683, State: Running, Role: LEADER
I20260808 10:48:41.650657 27490 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d70d268b1f49138033eb3420004683" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45457 } }
I20260808 10:48:41.651263 27483 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:48:41.652324 27492 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1d70d268b1f49138033eb3420004683" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d70d268b1f49138033eb3420004683" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45457 } } }
I20260808 10:48:41.652462 27492 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:41.652781 27493 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1d70d268b1f49138033eb3420004683. Latest consensus state: current_term: 1 leader_uuid: "c1d70d268b1f49138033eb3420004683" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d70d268b1f49138033eb3420004683" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45457 } } }
I20260808 10:48:41.652879 27493 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683 [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:41.653205 27501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:48:41.656380 27501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:48:41.662901 27501 catalog_manager.cc:1347] Generated new cluster ID: 8ac9ac09542e4a16944be5f44c4b60eb
I20260808 10:48:41.662992 27501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:48:41.674057 27501 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:48:41.675184 27501 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:48:41.690141 27501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683: Generated new TSK 0
I20260808 10:48:41.691159 27501 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:48:41.703091 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:45457
--builtin_ntp_servers=127.31.237.148:46599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:48:41.821779 27516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:41.822034 27516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:41.822072 27516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:41.826063 27516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:41.826119 27516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:41.826211 27516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:48:41.830999 27516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:46599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:45457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:41.832157 27516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:41.833616 27516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:48:41.843546 27516 server_base.cc:1034] running on GCE node
W20260808 10:48:41.843514 27524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:41.843580 27522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:41.843716 27526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:41.844141 27516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:41.844774 27516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:41.845984 27516 hybrid_clock.cc:648] HybridClock initialized: now 1786186121845968 us; error 27 us; skew 500 ppm
I20260808 10:48:41.847949 27516 webserver.cc:460] Webserver started at http://127.31.237.129:42551/ using document root <none> and password file <none>
I20260808 10:48:41.848572 27516 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:41.848675 27516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:41.848922 27516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:41.851241 27516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "900410a627084963afe5cb8c7ef16699"
format_stamp: "Formatted at 2026-08-08 10:48:41 on dist-test-slave-twwt"
server_key: "5194f1568b0767c7150465e714225e23"
server_key_iv: "90e7861139980463be479f87e3b423af"
server_key_version: "encryptionkey@0"
I20260808 10:48:41.851830 27516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "900410a627084963afe5cb8c7ef16699"
format_stamp: "Formatted at 2026-08-08 10:48:41 on dist-test-slave-twwt"
server_key: "5194f1568b0767c7150465e714225e23"
server_key_iv: "90e7861139980463be479f87e3b423af"
server_key_version: "encryptionkey@0"
I20260808 10:48:41.855798 27516 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 10:48:41.858430 27535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:41.859774 27516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:48:41.859959 27516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "900410a627084963afe5cb8c7ef16699"
format_stamp: "Formatted at 2026-08-08 10:48:41 on dist-test-slave-twwt"
server_key: "5194f1568b0767c7150465e714225e23"
server_key_iv: "90e7861139980463be479f87e3b423af"
server_key_version: "encryptionkey@0"
I20260808 10:48:41.860085 27516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:41.888912 27516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:41.889743 27516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:41.889976 27516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:41.890707 27516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:41.891760 27516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:41.891834 27516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:41.891911 27516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:41.891952 27516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:41.901897 27516 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:37935
I20260808 10:48:41.901917 27702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:37935 every 8 connection(s)
I20260808 10:48:41.902976 27516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:48:41.911477 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 27516
I20260808 10:48:41.911604 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:48:41.911973 32694 external_mini_cluster.cc:1428] Setting key 7bbedb7ca12d4ded3f2e4fcd3e087409
I20260808 10:48:41.913946 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:41.906921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:42189 (local address 127.31.237.190:45457)
0808 10:48:41.907093 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:41.907098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:41.907115 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:48:41.908690 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:41.908694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:41.908758 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:41.908840 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:41.909788 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.910332 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.911818 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.912110 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.912205 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:41.913432 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:41.913440 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:41.913442 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:41.913454 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:41.913475 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:41.913483 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:41.913564 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:41.913734 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 10:48:41.913800 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 10:48:41.914429 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:45457
--builtin_ntp_servers=127.31.237.148:46599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:48:41.914956 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:41.905645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:45457 (local address 127.31.237.129:42189)
0808 10:48:41.906729 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:41.906789 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:41.908458 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:41.909034 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:41.909042 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:41.909134 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:41.909643 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:41.909656 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.910475 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.910478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:41.911684 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:41.911697 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:41.912263 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:41.912268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:41.912474 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:41.913148 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:48:41.913150 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:41.913191 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:48:41.913257 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:41.913568 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:41.913580 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:41.913588 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:48:41.913733 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 10:48:41.914043 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":214,"threads_started":1}
I20260808 10:48:41.916317 27703 heartbeater.cc:347] Connected to a master server at 127.31.237.190:45457
I20260808 10:48:41.916603 27703 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:41.917258 27703 heartbeater.cc:511] Master 127.31.237.190:45457 requested a full tablet report, sending...
I20260808 10:48:41.919075 27420 ts_manager.cc:194] Registered new tserver with Master: 900410a627084963afe5cb8c7ef16699 (127.31.237.129:37935)
I20260808 10:48:41.920898 27420 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.237.129:42189
W20260808 10:48:42.036675 27712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:42.036970 27712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:42.037011 27712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:42.041333 27712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:42.041409 27712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:42.041469 27712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:48:42.046482 27712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:46599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:45457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:42.047726 27712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:42.049178 27712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:42.059685 27723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:42.059677 27720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:42.059677 27721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:42.059904 27712 server_base.cc:1034] running on GCE node
I20260808 10:48:42.060421 27712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:42.061074 27712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:42.062268 27712 hybrid_clock.cc:648] HybridClock initialized: now 1786186122062247 us; error 34 us; skew 500 ppm
I20260808 10:48:42.064453 27712 webserver.cc:460] Webserver started at http://127.31.237.130:41361/ using document root <none> and password file <none>
I20260808 10:48:42.065059 27712 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:42.065124 27712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:42.065388 27712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:42.067571 27712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "8dc80e1fb0d547538de5650f306c10a6"
format_stamp: "Formatted at 2026-08-08 10:48:42 on dist-test-slave-twwt"
server_key: "191e9b73b52420d3eaa43402f989d95c"
server_key_iv: "cf7eb90011d6536aa5a44644ab988e0b"
server_key_version: "encryptionkey@0"
I20260808 10:48:42.068166 27712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "8dc80e1fb0d547538de5650f306c10a6"
format_stamp: "Formatted at 2026-08-08 10:48:42 on dist-test-slave-twwt"
server_key: "191e9b73b52420d3eaa43402f989d95c"
server_key_iv: "cf7eb90011d6536aa5a44644ab988e0b"
server_key_version: "encryptionkey@0"
I20260808 10:48:42.071964 27712 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:48:42.074522 27732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:42.075876 27712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:48:42.076025 27712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "8dc80e1fb0d547538de5650f306c10a6"
format_stamp: "Formatted at 2026-08-08 10:48:42 on dist-test-slave-twwt"
server_key: "191e9b73b52420d3eaa43402f989d95c"
server_key_iv: "cf7eb90011d6536aa5a44644ab988e0b"
server_key_version: "encryptionkey@0"
I20260808 10:48:42.076138 27712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:42.086979 27712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:42.087723 27712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:42.087932 27712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:42.088585 27712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:42.089596 27712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:42.089643 27712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:42.089713 27712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:42.089753 27712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:42.099632 27712 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:45397
I20260808 10:48:42.099697 27914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:45397 every 8 connection(s)
I20260808 10:48:42.100621 27712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:48:42.103055 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 27712
I20260808 10:48:42.103152 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:48:42.103495 32694 external_mini_cluster.cc:1428] Setting key 3334b1599f0e0af9c08e1e28d3a3f376
I20260808 10:48:42.105630 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:45457
--builtin_ntp_servers=127.31.237.148:46599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:48:42.110587 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.102510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:41663 (local address 127.31.237.190:45457)
0808 10:48:42.102708 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:42.102712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:42.103974 (+ 1262us) server_negotiation.cc:407] Connection header received
0808 10:48:42.105696 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:42.105700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:42.105753 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:42.105832 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:42.106884 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.107408 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.108564 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.108789 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.108900 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.110064 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:42.110072 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:42.110075 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:42.110087 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:42.110108 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:42.110117 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:42.110191 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:42.110373 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 10:48:42.110442 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 10:48:42.111608 27917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.102999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:45457 (local address 127.31.237.130:41663)
0808 10:48:42.103790 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:42.103844 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:42.105486 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:42.106081 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:42.106088 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:42.106176 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:42.106744 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:42.106760 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.107542 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.107545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:42.108432 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:42.108443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.108956 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.108961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:42.109129 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.109776 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:48:42.109778 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:42.109819 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:48:42.109885 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:42.110211 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:42.110228 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:42.110233 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:48:42.110358 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 10:48:42.110678 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":205,"threads_started":1}
I20260808 10:48:42.112815 27915 heartbeater.cc:347] Connected to a master server at 127.31.237.190:45457
I20260808 10:48:42.113059 27915 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:42.113595 27915 heartbeater.cc:511] Master 127.31.237.190:45457 requested a full tablet report, sending...
I20260808 10:48:42.114750 27420 ts_manager.cc:194] Registered new tserver with Master: 8dc80e1fb0d547538de5650f306c10a6 (127.31.237.130:45397)
I20260808 10:48:42.115329 27420 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.237.130:41663
W20260808 10:48:42.226926 27918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:42.227200 27918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:42.227239 27918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:42.231304 27918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:42.231359 27918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:42.231451 27918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:48:42.236320 27918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:46599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:45457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:42.237461 27918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:42.238942 27918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:42.249276 27927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:42.249285 27931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:42.249794 27918 server_base.cc:1034] running on GCE node
W20260808 10:48:42.249440 27928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:42.250514 27918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:42.251539 27918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:42.252764 27918 hybrid_clock.cc:648] HybridClock initialized: now 1786186122252668 us; error 107 us; skew 500 ppm
I20260808 10:48:42.257283 27918 webserver.cc:460] Webserver started at http://127.31.237.131:41185/ using document root <none> and password file <none>
I20260808 10:48:42.258327 27918 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:42.258502 27918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:42.258886 27918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:42.261615 27918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "4a4cee08c45841f59b33397efc34a301"
format_stamp: "Formatted at 2026-08-08 10:48:42 on dist-test-slave-twwt"
server_key: "d8d5ac0e63a9ea6325f6060460cd7ddb"
server_key_iv: "36f436f6f9010bdf1aa520de242cbad5"
server_key_version: "encryptionkey@0"
I20260808 10:48:42.262235 27918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "4a4cee08c45841f59b33397efc34a301"
format_stamp: "Formatted at 2026-08-08 10:48:42 on dist-test-slave-twwt"
server_key: "d8d5ac0e63a9ea6325f6060460cd7ddb"
server_key_iv: "36f436f6f9010bdf1aa520de242cbad5"
server_key_version: "encryptionkey@0"
I20260808 10:48:42.266372 27918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 10:48:42.268905 27937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:42.270152 27918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:48:42.270289 27918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "4a4cee08c45841f59b33397efc34a301"
format_stamp: "Formatted at 2026-08-08 10:48:42 on dist-test-slave-twwt"
server_key: "d8d5ac0e63a9ea6325f6060460cd7ddb"
server_key_iv: "36f436f6f9010bdf1aa520de242cbad5"
server_key_version: "encryptionkey@0"
I20260808 10:48:42.270447 27918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:42.293004 27918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:42.294087 27918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:42.294356 27918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:42.295295 27918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:42.296736 27918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:42.296824 27918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:42.296885 27918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:42.296906 27918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:42.311755 27918 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:45155
I20260808 10:48:42.311828 28111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:45155 every 8 connection(s)
I20260808 10:48:42.312798 27918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:48:42.313776 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 27918
I20260808 10:48:42.313910 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:48:42.314275 32694 external_mini_cluster.cc:1428] Setting key f2ff86244983c0490fdc2c2e4ae757f1
I20260808 10:48:42.327227 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.318101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:49587 (local address 127.31.237.190:45457)
0808 10:48:42.318234 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:42.318238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:42.318253 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:48:42.320526 (+ 2273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:42.320529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:42.320579 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:42.320639 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:42.322037 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.322561 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.324560 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.324920 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.325031 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.326602 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:42.326628 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:42.326631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:42.326649 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:42.326674 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:42.326684 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:42.326782 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:42.326994 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 10:48:42.327053 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:48:42.328475 28115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.317073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:45457 (local address 127.31.237.131:49587)
0808 10:48:42.318054 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:42.318123 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:42.320239 (+ 2116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:42.320866 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:42.320880 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:42.321021 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:42.321849 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:42.321874 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.322732 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.322739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:42.324407 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:42.324424 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.325121 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.325128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:42.325407 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.326242 (+ 835us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:48:42.326245 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:42.326312 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:48:42.326425 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:42.326806 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:42.326819 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:42.326828 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:48:42.326989 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 10:48:42.327324 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":170,"threads_started":1}
I20260808 10:48:42.329761 28113 heartbeater.cc:347] Connected to a master server at 127.31.237.190:45457
I20260808 10:48:42.330067 28113 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:42.330768 28113 heartbeater.cc:511] Master 127.31.237.190:45457 requested a full tablet report, sending...
I20260808 10:48:42.331849 27420 ts_manager.cc:194] Registered new tserver with Master: 4a4cee08c45841f59b33397efc34a301 (127.31.237.131:45155)
I20260808 10:48:42.332437 27420 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.237.131:49587
I20260808 10:48:42.343118 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:48:42.348914 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.345137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59384 (local address 127.31.237.190:45457)
0808 10:48:42.345327 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:42.345333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:42.345449 (+ 116us) server_negotiation.cc:407] Connection header received
0808 10:48:42.345552 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:42.345554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:42.345598 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:42.345669 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:42.345942 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.346455 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.347348 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.347574 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.347660 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.348450 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:42.348453 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:42.348455 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:42.348469 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:42.348489 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:42.348496 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:42.348555 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:42.348651 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 10:48:42.348702 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 10:48:42.351774 27420 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:48:42.355170 27420 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:48:42.370923 28135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.364409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:45397 (local address 127.0.0.1:51066)
0808 10:48:42.365453 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:42.365466 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:42.365542 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:42.366444 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:42.366448 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:42.366469 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:42.366566 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:42.366572 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.367968 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.367975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:42.369283 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:42.369294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.370290 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.370294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:42.370430 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.370478 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 10:48:42.370589 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 10:48:42.370739 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":980,"mutex_wait_us":53,"thread_start_us":108,"threads_started":1}
I20260808 10:48:42.371376 28130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.364651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34002 (local address 127.31.237.131:45155)
0808 10:48:42.364999 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:42.365013 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:42.365584 (+ 571us) server_negotiation.cc:407] Connection header received
0808 10:48:42.365681 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:42.365686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:42.365905 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:42.366086 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:42.366428 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.367715 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.369756 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.370342 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.370459 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.370998 (+ 539us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:42.371056 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 10:48:42.371157 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":140,"threads_started":1}
I20260808 10:48:42.371670 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.364283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:45155 (local address 127.0.0.1:34002)
0808 10:48:42.365209 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:42.365267 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:42.365423 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:42.366086 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:42.366092 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:42.366117 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:42.366281 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:42.366289 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.367870 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.367876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:42.369598 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:42.369610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.370698 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.370702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:42.370777 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.370800 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:48:42.370873 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:48:42.370933 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":187,"thread_start_us":96,"threads_started":1}
I20260808 10:48:42.371675 28137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.365282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35592 (local address 127.31.237.129:37935)
0808 10:48:42.365696 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:42.365704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:42.366386 (+ 682us) server_negotiation.cc:407] Connection header received
0808 10:48:42.366515 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:42.366520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:42.366739 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:42.366890 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:42.367232 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.368008 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.369733 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.370517 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.370667 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.370825 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:42.371390 (+ 565us) server_negotiation.cc:299] Negotiation successful
0808 10:48:42.371503 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":118,"threads_started":1}
I20260808 10:48:42.371865 28133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.365083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51066 (local address 127.31.237.130:45397)
0808 10:48:42.365835 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:42.365845 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:42.365865 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:48:42.365932 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:42.365936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:42.366127 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:42.366281 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:42.366743 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.367846 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.369410 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.370151 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.370282 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.370477 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:42.370652 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 10:48:42.370764 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":153,"threads_started":1}
I20260808 10:48:42.372210 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.365126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:37935 (local address 127.0.0.1:35592)
0808 10:48:42.366269 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:42.366284 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:42.366402 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:42.367026 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:42.367028 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:42.367044 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:42.367114 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:42.367119 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.368127 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.368131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:42.369598 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:42.369610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.371153 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.371157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:42.371241 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.371264 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:48:42.371338 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 10:48:42.371412 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1073,"mutex_wait_us":126,"thread_start_us":117,"threads_started":1}
I20260808 10:48:42.375670 27598 tablet_service.cc:1467] Processing CreateTablet for tablet 868c425e67f3487fa18d4f9d2b46a810 (DEFAULT_TABLE table=test-table [id=3fd52a977a3f4f7e9888e7464e89d6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:42.375545 28006 tablet_service.cc:1467] Processing CreateTablet for tablet 868c425e67f3487fa18d4f9d2b46a810 (DEFAULT_TABLE table=test-table [id=3fd52a977a3f4f7e9888e7464e89d6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:42.376824 28006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 868c425e67f3487fa18d4f9d2b46a810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:42.376824 27598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 868c425e67f3487fa18d4f9d2b46a810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:42.376751 27807 tablet_service.cc:1467] Processing CreateTablet for tablet 868c425e67f3487fa18d4f9d2b46a810 (DEFAULT_TABLE table=test-table [id=3fd52a977a3f4f7e9888e7464e89d6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:42.377873 27807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 868c425e67f3487fa18d4f9d2b46a810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:42.385063 28138 tablet_bootstrap.cc:492] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6: Bootstrap starting.
I20260808 10:48:42.386696 28140 tablet_bootstrap.cc:492] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301: Bootstrap starting.
I20260808 10:48:42.387104 28142 tablet_bootstrap.cc:492] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699: Bootstrap starting.
I20260808 10:48:42.388132 28138 tablet_bootstrap.cc:654] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:42.388932 28142 tablet_bootstrap.cc:654] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:42.388931 28138 log.cc:824] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:42.389590 28142 log.cc:824] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:42.391268 28138 tablet_bootstrap.cc:492] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6: No bootstrap required, opened a new log
I20260808 10:48:42.391434 28142 tablet_bootstrap.cc:492] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699: No bootstrap required, opened a new log
I20260808 10:48:42.391486 28138 ts_tablet_manager.cc:1397] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 10:48:42.391669 28142 ts_tablet_manager.cc:1397] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 10:48:42.391988 28140 tablet_bootstrap.cc:654] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:42.393021 28140 log.cc:824] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:42.395401 28140 tablet_bootstrap.cc:492] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301: No bootstrap required, opened a new log
I20260808 10:48:42.395470 28142 raft_consensus.cc:348] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4cee08c45841f59b33397efc34a301" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45155 } } peers { permanent_uuid: "8dc80e1fb0d547538de5650f306c10a6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 45397 } } peers { permanent_uuid: "900410a627084963afe5cb8c7ef16699" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37935 } }
I20260808 10:48:42.395660 28140 ts_tablet_manager.cc:1397] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 10:48:42.395730 28142 raft_consensus.cc:374] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:42.395807 28142 raft_consensus.cc:729] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 900410a627084963afe5cb8c7ef16699, State: Initialized, Role: FOLLOWER
I20260808 10:48:42.396301 28142 consensus_queue.cc:260] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4cee08c45841f59b33397efc34a301" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45155 } } peers { permanent_uuid: "8dc80e1fb0d547538de5650f306c10a6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 45397 } } peers { permanent_uuid: "900410a627084963afe5cb8c7ef16699" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37935 } }
I20260808 10:48:42.396658 28138 raft_consensus.cc:348] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4cee08c45841f59b33397efc34a301" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45155 } } peers { permanent_uuid: "8dc80e1fb0d547538de5650f306c10a6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 45397 } } peers { permanent_uuid: "900410a627084963afe5cb8c7ef16699" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37935 } }
I20260808 10:48:42.396958 28138 raft_consensus.cc:374] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:42.397028 28138 raft_consensus.cc:729] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dc80e1fb0d547538de5650f306c10a6, State: Initialized, Role: FOLLOWER
I20260808 10:48:42.397032 27703 heartbeater.cc:503] Master 127.31.237.190:45457 was elected leader, sending a full tablet report...
I20260808 10:48:42.397527 28138 consensus_queue.cc:260] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4cee08c45841f59b33397efc34a301" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45155 } } peers { permanent_uuid: "8dc80e1fb0d547538de5650f306c10a6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 45397 } } peers { permanent_uuid: "900410a627084963afe5cb8c7ef16699" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37935 } }
I20260808 10:48:42.398483 27915 heartbeater.cc:503] Master 127.31.237.190:45457 was elected leader, sending a full tablet report...
I20260808 10:48:42.398680 28138 ts_tablet_manager.cc:1428] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 10:48:42.399752 28142 ts_tablet_manager.cc:1428] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260808 10:48:42.400431 28140 raft_consensus.cc:348] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4cee08c45841f59b33397efc34a301" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45155 } } peers { permanent_uuid: "8dc80e1fb0d547538de5650f306c10a6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 45397 } } peers { permanent_uuid: "900410a627084963afe5cb8c7ef16699" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37935 } }
I20260808 10:48:42.400749 28140 raft_consensus.cc:374] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:42.400856 28140 raft_consensus.cc:729] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a4cee08c45841f59b33397efc34a301, State: Initialized, Role: FOLLOWER
I20260808 10:48:42.401445 28140 consensus_queue.cc:260] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4cee08c45841f59b33397efc34a301" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45155 } } peers { permanent_uuid: "8dc80e1fb0d547538de5650f306c10a6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 45397 } } peers { permanent_uuid: "900410a627084963afe5cb8c7ef16699" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37935 } }
I20260808 10:48:42.402187 28113 heartbeater.cc:503] Master 127.31.237.190:45457 was elected leader, sending a full tablet report...
I20260808 10:48:42.402470 28140 ts_tablet_manager.cc:1428] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260808 10:48:42.404173 27708 tablet.cc:2369] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:48:42.454846 28148 raft_consensus.cc:482] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:48:42.455072 28148 raft_consensus.cc:504] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4cee08c45841f59b33397efc34a301" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45155 } } peers { permanent_uuid: "8dc80e1fb0d547538de5650f306c10a6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 45397 } } peers { permanent_uuid: "900410a627084963afe5cb8c7ef16699" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37935 } }
I20260808 10:48:42.456238 28148 leader_election.cc:288] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a4cee08c45841f59b33397efc34a301 (127.31.237.131:45155), 900410a627084963afe5cb8c7ef16699 (127.31.237.129:37935)
I20260808 10:48:42.460492 27917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.456524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:45155 (local address 127.31.237.130:35487)
0808 10:48:42.456679 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:42.456697 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:42.456812 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:42.457133 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:42.457136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:42.457159 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:42.457315 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:42.457323 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.458229 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.458232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:42.459220 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:42.459231 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.460017 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.460021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:42.460123 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.460150 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:48:42.460235 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:48:42.460325 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 10:48:42.460493 28130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.456663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:35487 (local address 127.31.237.131:45155)
0808 10:48:42.456815 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:42.456820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:42.456837 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:48:42.456932 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:42.456935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:42.457006 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:42.457098 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:42.457474 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.458118 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.459351 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.459892 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.460000 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.460043 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:42.460257 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 10:48:42.460352 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 10:48:42.460803 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.456524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:37935 (local address 127.31.237.130:53899)
0808 10:48:42.457139 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:42.457152 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:42.457260 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:42.457545 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:42.457548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:42.457566 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:42.457699 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:42.457705 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.458697 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.458700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:42.459575 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:42.459586 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.460356 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.460359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:42.460466 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.460489 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:48:42.460594 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 10:48:42.460663 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":201,"thread_start_us":191,"threads_started":1}
I20260808 10:48:42.460958 28137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.456650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:53899 (local address 127.31.237.129:37935)
0808 10:48:42.456818 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:42.456825 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:42.457241 (+ 416us) server_negotiation.cc:407] Connection header received
0808 10:48:42.457359 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:42.457365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:42.457463 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:42.457522 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:42.457832 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.458525 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.459706 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.460231 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.460341 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.460386 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:42.460744 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 10:48:42.460835 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:48:42.461282 28032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "868c425e67f3487fa18d4f9d2b46a810" candidate_uuid: "8dc80e1fb0d547538de5650f306c10a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a4cee08c45841f59b33397efc34a301" is_pre_election: true
I20260808 10:48:42.461596 27625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "868c425e67f3487fa18d4f9d2b46a810" candidate_uuid: "8dc80e1fb0d547538de5650f306c10a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "900410a627084963afe5cb8c7ef16699" is_pre_election: true
I20260808 10:48:42.461643 28032 raft_consensus.cc:2454] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dc80e1fb0d547538de5650f306c10a6 in term 0.
I20260808 10:48:42.461907 27625 raft_consensus.cc:2454] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dc80e1fb0d547538de5650f306c10a6 in term 0.
I20260808 10:48:42.462210 27740 leader_election.cc:302] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a4cee08c45841f59b33397efc34a301, 8dc80e1fb0d547538de5650f306c10a6; no voters:
I20260808 10:48:42.462587 28148 raft_consensus.cc:2781] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:48:42.462826 28148 raft_consensus.cc:482] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:48:42.462913 28148 raft_consensus.cc:3037] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:42.464222 28148 raft_consensus.cc:504] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4cee08c45841f59b33397efc34a301" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45155 } } peers { permanent_uuid: "8dc80e1fb0d547538de5650f306c10a6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 45397 } } peers { permanent_uuid: "900410a627084963afe5cb8c7ef16699" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37935 } }
I20260808 10:48:42.464696 28148 leader_election.cc:288] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4a4cee08c45841f59b33397efc34a301 (127.31.237.131:45155), 900410a627084963afe5cb8c7ef16699 (127.31.237.129:37935)
I20260808 10:48:42.465081 27625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "868c425e67f3487fa18d4f9d2b46a810" candidate_uuid: "8dc80e1fb0d547538de5650f306c10a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "900410a627084963afe5cb8c7ef16699"
I20260808 10:48:42.465241 27625 raft_consensus.cc:3037] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:42.466409 27625 raft_consensus.cc:2454] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dc80e1fb0d547538de5650f306c10a6 in term 1.
I20260808 10:48:42.466818 27736 leader_election.cc:302] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dc80e1fb0d547538de5650f306c10a6, 900410a627084963afe5cb8c7ef16699; no voters:
I20260808 10:48:42.466796 28032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "868c425e67f3487fa18d4f9d2b46a810" candidate_uuid: "8dc80e1fb0d547538de5650f306c10a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a4cee08c45841f59b33397efc34a301"
I20260808 10:48:42.466964 28032 raft_consensus.cc:3037] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:42.467020 28148 raft_consensus.cc:2781] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:42.467289 28148 raft_consensus.cc:686] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [term 1 LEADER]: Becoming Leader. State: Replica: 8dc80e1fb0d547538de5650f306c10a6, State: Running, Role: LEADER
I20260808 10:48:42.467686 28148 consensus_queue.cc:237] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4cee08c45841f59b33397efc34a301" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45155 } } peers { permanent_uuid: "8dc80e1fb0d547538de5650f306c10a6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 45397 } } peers { permanent_uuid: "900410a627084963afe5cb8c7ef16699" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37935 } }
I20260808 10:48:42.468418 28032 raft_consensus.cc:2454] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dc80e1fb0d547538de5650f306c10a6 in term 1.
I20260808 10:48:42.471314 27427 catalog_manager.cc:5495] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8dc80e1fb0d547538de5650f306c10a6 (127.31.237.130). New cstate: current_term: 1 leader_uuid: "8dc80e1fb0d547538de5650f306c10a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a4cee08c45841f59b33397efc34a301" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dc80e1fb0d547538de5650f306c10a6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 45397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "900410a627084963afe5cb8c7ef16699" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37935 } health_report { overall_health: UNKNOWN } } }
I20260808 10:48:42.500866 28133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.497393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51076 (local address 127.31.237.130:45397)
0808 10:48:42.497527 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:42.497531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:42.497544 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:48:42.497623 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:42.497625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:42.497681 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:42.497778 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:48:42.498012 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.498510 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.499361 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:42.499574 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:42.499659 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:42.499939 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:48:42.500371 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:48:42.500464 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:42.500631 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 10:48:42.500690 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:48:42.507593 27625 raft_consensus.cc:1261] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699 [term 1 FOLLOWER]: Refusing update from remote peer 8dc80e1fb0d547538de5650f306c10a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:48:42.507628 28032 raft_consensus.cc:1261] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301 [term 1 FOLLOWER]: Refusing update from remote peer 8dc80e1fb0d547538de5650f306c10a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:48:42.508391 28151 consensus_queue.cc:1035] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "900410a627084963afe5cb8c7ef16699" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 37935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:48:42.508581 28148 consensus_queue.cc:1035] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a4cee08c45841f59b33397efc34a301" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 45155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:48:42.517725 28156 mvcc.cc:205] Tried to move back new op lower bound from 7316218357782138880 to 7316218357631279104. Current Snapshot: MvccSnapshot[applied={T|T < 7316218357782138880}]
I20260808 10:48:42.526857 27420 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:48:42.527040 27420 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:48:42.529551 27420 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1d70d268b1f49138033eb3420004683: Sending DeleteTablet for 3 replicas of tablet 868c425e67f3487fa18d4f9d2b46a810
I20260808 10:48:42.530362 28006 tablet_service.cc:1514] Processing DeleteTablet for tablet 868c425e67f3487fa18d4f9d2b46a810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:42 UTC) from {username='slave'} at 127.0.0.1:34002
I20260808 10:48:42.530380 27807 tablet_service.cc:1514] Processing DeleteTablet for tablet 868c425e67f3487fa18d4f9d2b46a810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:42 UTC) from {username='slave'} at 127.0.0.1:51066
I20260808 10:48:42.530404 27598 tablet_service.cc:1514] Processing DeleteTablet for tablet 868c425e67f3487fa18d4f9d2b46a810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:42 UTC) from {username='slave'} at 127.0.0.1:35592
I20260808 10:48:42.530889 28170 tablet_replica.cc:330] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6: stopping tablet replica
I20260808 10:48:42.531031 28172 tablet_replica.cc:330] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301: stopping tablet replica
I20260808 10:48:42.531268 28171 tablet_replica.cc:330] T 868c425e67f3487fa18d4f9d2b46a810 P 900410a627084963afe5cb8c7ef16699: stopping tablet replica
I20260808 10:48:42.531291 28170 raft_consensus.cc:2229] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:48:42.531402 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 27516
I20260808 10:48:42.531636 28172 raft_consensus.cc:2229] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:48:42.531906 28172 raft_consensus.cc:2258] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:42.534243 28170 raft_consensus.cc:2258] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:42.538519 28170 ts_tablet_manager.cc:1905] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:42.542698 28170 ts_tablet_manager.cc:1918] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:42.542805 28170 log.cc:1196] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/868c425e67f3487fa18d4f9d2b46a810
I20260808 10:48:42.543139 28170 ts_tablet_manager.cc:1939] T 868c425e67f3487fa18d4f9d2b46a810 P 8dc80e1fb0d547538de5650f306c10a6: Deleting consensus metadata
I20260808 10:48:42.543182 28172 ts_tablet_manager.cc:1905] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:42.544574 27404 catalog_manager.cc:4841] TS 8dc80e1fb0d547538de5650f306c10a6 (127.31.237.130:45397): tablet 868c425e67f3487fa18d4f9d2b46a810 (table test-table [id=3fd52a977a3f4f7e9888e7464e89d6d8]) successfully deleted
I20260808 10:48:42.546010 28172 ts_tablet_manager.cc:1918] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:42.546106 28172 log.cc:1196] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/868c425e67f3487fa18d4f9d2b46a810
I20260808 10:48:42.546427 28172 ts_tablet_manager.cc:1939] T 868c425e67f3487fa18d4f9d2b46a810 P 4a4cee08c45841f59b33397efc34a301: Deleting consensus metadata
I20260808 10:48:42.547791 27409 catalog_manager.cc:4841] TS 4a4cee08c45841f59b33397efc34a301 (127.31.237.131:45155): tablet 868c425e67f3487fa18d4f9d2b46a810 (table test-table [id=3fd52a977a3f4f7e9888e7464e89d6d8]) successfully deleted
W20260808 10:48:42.553222 27406 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:37935 (error 108)
I20260808 10:48:42.553495 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 27712
I20260808 10:48:42.559866 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:42.558593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:37935 (local address 127.0.0.1:35594)
0808 10:48:42.559658 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:42.559756 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:37935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":934}
W20260808 10:48:42.562872 27406 catalog_manager.cc:4584] TS 900410a627084963afe5cb8c7ef16699 (127.31.237.129:37935): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 868c425e67f3487fa18d4f9d2b46a810: Network error: Client connection negotiation failed: client connection to 127.31.237.129:37935: connect: Connection refused (error 111)
I20260808 10:48:42.573210 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 27918
I20260808 10:48:42.589483 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 27383
2026-08-08T10:48:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1187 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:48:42 dist-test-slave-twwt krb5kdc[28179](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:48:42 dist-test-slave-twwt krb5kdc[28179](info): set up 2 sockets
Aug 08 10:48:42 dist-test-slave-twwt krb5kdc[28179](info): commencing operation
krb5kdc: starting...
W20260808 10:48:43.841385 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:48:43 dist-test-slave-twwt krb5kdc[28179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:48:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:48:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:48:43.977536 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:43225
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:43225
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:44.094478 28201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:44.094770 28201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:44.094812 28201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:44.098714 28201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:48:44.098783 28201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:44.098804 28201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:44.098821 28201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:48:44.098836 28201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:48:44.104130 28201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:43225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:43225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:44.105257 28201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:44.106331 28201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:48:44.115907 28201 server_base.cc:1034] running on GCE node
W20260808 10:48:44.115854 28208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:44.115886 28215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:44.116254 28209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:44.116952 28201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:44.118160 28201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:44.119329 28201 hybrid_clock.cc:648] HybridClock initialized: now 1786186124119308 us; error 33 us; skew 500 ppm
Aug 08 10:48:44 dist-test-slave-twwt krb5kdc[28179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186124, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:44.126299 28201 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:48:44.127331 28201 webserver.cc:460] Webserver started at http://127.31.237.190:37151/ using document root <none> and password file <none>
I20260808 10:48:44.127929 28201 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:44.127991 28201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:44.128232 28201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:44.130342 28201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "1e271d7a16e94f33be99a2d131c1eeeb"
format_stamp: "Formatted at 2026-08-08 10:48:44 on dist-test-slave-twwt"
server_key: "5032324f880967144af9b04418acd154"
server_key_iv: "b242f2dd82955413ce3dcac20d85e9ad"
server_key_version: "encryptionkey@0"
I20260808 10:48:44.130977 28201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "1e271d7a16e94f33be99a2d131c1eeeb"
format_stamp: "Formatted at 2026-08-08 10:48:44 on dist-test-slave-twwt"
server_key: "5032324f880967144af9b04418acd154"
server_key_iv: "b242f2dd82955413ce3dcac20d85e9ad"
server_key_version: "encryptionkey@0"
I20260808 10:48:44.134902 28201 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:48:44.137434 28226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:44.138825 28201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:48:44.138970 28201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "1e271d7a16e94f33be99a2d131c1eeeb"
format_stamp: "Formatted at 2026-08-08 10:48:44 on dist-test-slave-twwt"
server_key: "5032324f880967144af9b04418acd154"
server_key_iv: "b242f2dd82955413ce3dcac20d85e9ad"
server_key_version: "encryptionkey@0"
I20260808 10:48:44.139081 28201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:44.172322 28201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:44.175705 28201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:44.175930 28201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:44.183879 28201 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:43225
I20260808 10:48:44.183907 28303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:43225 every 8 connection(s)
I20260808 10:48:44.184994 28201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:48:44.185392 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 28201
I20260808 10:48:44.185580 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:48:44.185909 32694 external_mini_cluster.cc:1428] Setting key 7a181865a2234d3e60d39a6e3286fb7e
I20260808 10:48:44.188660 28305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 10:48:44 dist-test-slave-twwt krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:48:44.196036 28305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb: Bootstrap starting.
I20260808 10:48:44.198931 28305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:44.199967 28305 log.cc:824] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:44.201958 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:44.187453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42782 (local address 127.31.237.190:43225)
0808 10:48:44.187898 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:44.187919 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:44.187965 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:48:44.188886 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:44.188911 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:44.189337 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:44.189755 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:44.190662 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.192199 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:44.193059 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.193746 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:44.193926 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:44.196716 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:44.196723 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:44.196739 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:44.196772 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:44.198724 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:44.199177 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:44.199185 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:44.199192 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:44.199339 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:44.199633 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:44.199636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:44.199638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:44.200182 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:44.200397 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:44.200672 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 10:48:44.200988 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":123,"threads_started":1}
I20260808 10:48:44.202395 28305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb: No bootstrap required, opened a new log
I20260808 10:48:44.206336 28305 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e271d7a16e94f33be99a2d131c1eeeb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43225 } }
I20260808 10:48:44.206579 28305 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:44.206672 28305 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e271d7a16e94f33be99a2d131c1eeeb, State: Initialized, Role: FOLLOWER
I20260808 10:48:44.207104 28305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e271d7a16e94f33be99a2d131c1eeeb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43225 } }
I20260808 10:48:44.207229 28305 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:48:44.207269 28305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:48:44.207334 28305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:44.208467 28305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e271d7a16e94f33be99a2d131c1eeeb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43225 } }
I20260808 10:48:44.208827 28305 leader_election.cc:302] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e271d7a16e94f33be99a2d131c1eeeb; no voters:
I20260808 10:48:44.209164 28305 leader_election.cc:288] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:48:44.209384 28315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:44.209717 28315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [term 1 LEADER]: Becoming Leader. State: Replica: 1e271d7a16e94f33be99a2d131c1eeeb, State: Running, Role: LEADER
I20260808 10:48:44.210033 28315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e271d7a16e94f33be99a2d131c1eeeb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43225 } }
I20260808 10:48:44.210531 28305 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:48:44.212062 28316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e271d7a16e94f33be99a2d131c1eeeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e271d7a16e94f33be99a2d131c1eeeb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43225 } } }
I20260808 10:48:44.212065 28319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e271d7a16e94f33be99a2d131c1eeeb. Latest consensus state: current_term: 1 leader_uuid: "1e271d7a16e94f33be99a2d131c1eeeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e271d7a16e94f33be99a2d131c1eeeb" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43225 } } }
I20260808 10:48:44.212251 28316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:44.212251 28319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:44.212684 28327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:48:44.216100 28327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:48:44.222245 28327 catalog_manager.cc:1347] Generated new cluster ID: 24cfa4e19a9048519b4e20f5771125e7
I20260808 10:48:44.222334 28327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:48:44.236965 28327 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:48:44.237933 28327 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:48:44.249701 28327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb: Generated new TSK 0
I20260808 10:48:44.250494 28327 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:48:44.285699 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:43225
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:44.403479 28343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:44.403765 28343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:44.403811 28343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:44.407886 28343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:44.407958 28343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:44.408048 28343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:48:44.413887 28343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:43225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:44.415043 28343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:44.416119 28343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:44.426074 28349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:44.426090 28348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:44.426092 28351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:44.426542 28343 server_base.cc:1034] running on GCE node
I20260808 10:48:44.426975 28343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:44.427644 28343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:44.428836 28343 hybrid_clock.cc:648] HybridClock initialized: now 1786186124428809 us; error 45 us; skew 500 ppm
Aug 08 10:48:44 dist-test-slave-twwt krb5kdc[28179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186124, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:44.431670 28343 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:48:44.432562 28343 webserver.cc:460] Webserver started at http://127.31.237.129:42943/ using document root <none> and password file <none>
I20260808 10:48:44.433168 28343 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:44.433261 28343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:44.433511 28343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:44.435650 28343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "803dd845e2f241dcb159d17217d788f8"
format_stamp: "Formatted at 2026-08-08 10:48:44 on dist-test-slave-twwt"
server_key: "7d1ca478198a4ef0004c51727f7a29e1"
server_key_iv: "7feba0295cfc01b8a9b9969b8fabf60b"
server_key_version: "encryptionkey@0"
I20260808 10:48:44.436232 28343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "803dd845e2f241dcb159d17217d788f8"
format_stamp: "Formatted at 2026-08-08 10:48:44 on dist-test-slave-twwt"
server_key: "7d1ca478198a4ef0004c51727f7a29e1"
server_key_iv: "7feba0295cfc01b8a9b9969b8fabf60b"
server_key_version: "encryptionkey@0"
I20260808 10:48:44.440187 28343 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:48:44.442965 28363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:44.444389 28343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:48:44.444532 28343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "803dd845e2f241dcb159d17217d788f8"
format_stamp: "Formatted at 2026-08-08 10:48:44 on dist-test-slave-twwt"
server_key: "7d1ca478198a4ef0004c51727f7a29e1"
server_key_iv: "7feba0295cfc01b8a9b9969b8fabf60b"
server_key_version: "encryptionkey@0"
I20260808 10:48:44.444657 28343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:44.486037 28343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:44.489696 28343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:44.489928 28343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:44.490772 28343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:44.491673 28343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:44.491732 28343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:44.491801 28343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:44.491830 28343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:44.500988 28343 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:45519
I20260808 10:48:44.501011 28538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:45519 every 8 connection(s)
I20260808 10:48:44.501942 28343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:48:44.503561 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 28343
I20260808 10:48:44.503723 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:48:44.504139 32694 external_mini_cluster.cc:1428] Setting key 57368e5233a064da2a667b58555003cb
Aug 08 10:48:44 dist-test-slave-twwt krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186124, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:48:44.517736 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:44.503880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:55535 (local address 127.31.237.190:43225)
0808 10:48:44.504050 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:44.504055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:44.505132 (+ 1077us) server_negotiation.cc:407] Connection header received
0808 10:48:44.506994 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:44.506997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:44.507045 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:44.507119 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:44.508727 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.509282 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:44.510626 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.510887 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:44.511004 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:44.513897 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:44.513901 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:44.513905 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:44.513937 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:44.515512 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:44.516118 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:44.516121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:44.516122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:44.516202 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:44.516554 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:44.516558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:44.516560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:44.516799 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:44.516909 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:44.517427 (+ 518us) server_negotiation.cc:299] Negotiation successful
0808 10:48:44.517564 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
I20260808 10:48:44.518838 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:44.504408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:43225 (local address 127.31.237.129:55535)
0808 10:48:44.504982 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:44.505028 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:44.506730 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:44.507307 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:44.507318 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:44.507890 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:44.508555 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:44.508570 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.509436 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:44.509445 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:44.510477 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:44.510487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.511064 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:44.511068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:44.511276 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:44.511990 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:44.512008 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:44.513714 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:44.515633 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:44.515640 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:44.515657 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:44.515999 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:44.516296 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:44.516300 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:44.516302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:44.516454 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:44.516909 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:44.516918 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:44.517187 (+ 269us) client_negotiation.cc:769] Sending connection context
0808 10:48:44.517384 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 10:48:44.517728 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":134,"threads_started":1}
I20260808 10:48:44.520237 28539 heartbeater.cc:347] Connected to a master server at 127.31.237.190:43225
I20260808 10:48:44.520522 28539 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:44.521198 28539 heartbeater.cc:511] Master 127.31.237.190:43225 requested a full tablet report, sending...
I20260808 10:48:44.523100 28257 ts_manager.cc:194] Registered new tserver with Master: 803dd845e2f241dcb159d17217d788f8 (127.31.237.129:45519)
I20260808 10:48:44.524847 28257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:55535
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:48:44.538034 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:43225
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:44.657780 28552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:44.658066 28552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:44.658113 28552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:44.662189 28552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:44.662261 28552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:44.662322 28552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:48:44.667694 28552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:43225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:44.668821 28552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:44.669893 28552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:44.680236 28565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:44.680217 28564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:44.680215 28567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:44.680416 28552 server_base.cc:1034] running on GCE node
I20260808 10:48:44.680924 28552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:44.681636 28552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:44.682827 28552 hybrid_clock.cc:648] HybridClock initialized: now 1786186124682784 us; error 53 us; skew 500 ppm
Aug 08 10:48:44 dist-test-slave-twwt krb5kdc[28179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186124, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:44.686357 28552 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:48:44.687374 28552 webserver.cc:460] Webserver started at http://127.31.237.130:41291/ using document root <none> and password file <none>
I20260808 10:48:44.687983 28552 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:44.688076 28552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:44.688305 28552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:44.690438 28552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "3d5552b276fb4a70899c11237323f8b6"
format_stamp: "Formatted at 2026-08-08 10:48:44 on dist-test-slave-twwt"
server_key: "0199ae8dec1a622449107fbd7371f22a"
server_key_iv: "545c00a1ca36ee3b5bc7045c8ebf2767"
server_key_version: "encryptionkey@0"
I20260808 10:48:44.691123 28552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "3d5552b276fb4a70899c11237323f8b6"
format_stamp: "Formatted at 2026-08-08 10:48:44 on dist-test-slave-twwt"
server_key: "0199ae8dec1a622449107fbd7371f22a"
server_key_iv: "545c00a1ca36ee3b5bc7045c8ebf2767"
server_key_version: "encryptionkey@0"
I20260808 10:48:44.695083 28552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:48:44.697649 28578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:44.699018 28552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:48:44.699153 28552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "3d5552b276fb4a70899c11237323f8b6"
format_stamp: "Formatted at 2026-08-08 10:48:44 on dist-test-slave-twwt"
server_key: "0199ae8dec1a622449107fbd7371f22a"
server_key_iv: "545c00a1ca36ee3b5bc7045c8ebf2767"
server_key_version: "encryptionkey@0"
I20260808 10:48:44.699266 28552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:44.714844 28552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:44.717943 28552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:44.718133 28552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:44.718782 28552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:44.719647 28552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:44.719717 28552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:44.719789 28552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:44.719841 28552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:44.729027 28552 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:33551
I20260808 10:48:44.729061 28739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:33551 every 8 connection(s)
I20260808 10:48:44.729918 28552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:48:44.735710 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 28552
I20260808 10:48:44.735893 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:48:44.736224 32694 external_mini_cluster.cc:1428] Setting key 2bb384a7c630480e633a5597595bd800
Aug 08 10:48:44 dist-test-slave-twwt krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186124, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:48:44.745159 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:44.731912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:38563 (local address 127.31.237.190:43225)
0808 10:48:44.732078 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:44.732082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:44.732952 (+ 870us) server_negotiation.cc:407] Connection header received
0808 10:48:44.734257 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:44.734260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:44.734307 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:44.734393 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:44.735650 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.736448 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:44.737807 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.738117 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:44.738239 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:44.740964 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:44.740967 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:44.740969 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:44.740993 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:44.742532 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:44.743276 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:44.743279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:44.743281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:44.743397 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:44.743880 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:44.743882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:44.743884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:44.744073 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:44.744151 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:44.744917 (+ 766us) server_negotiation.cc:299] Negotiation successful
0808 10:48:44.745030 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 10:48:44.746495 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:44.732281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:43225 (local address 127.31.237.130:38563)
0808 10:48:44.732774 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:44.732818 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:44.734060 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:44.734562 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:44.734568 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:44.735009 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:44.735497 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:44.735511 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.736572 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:44.736575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:44.737705 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:44.737716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.738297 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:44.738301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:44.738452 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:44.739137 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:44.739154 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:44.740776 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:44.742678 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:44.742688 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:44.742708 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:44.743177 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:44.743508 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:44.743515 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:44.743519 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:44.743751 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:44.744191 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:44.744202 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:44.744588 (+ 386us) client_negotiation.cc:769] Sending connection context
0808 10:48:44.744849 (+ 261us) client_negotiation.cc:240] Negotiation successful
0808 10:48:44.745283 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":117,"threads_started":1}
I20260808 10:48:44.747880 28740 heartbeater.cc:347] Connected to a master server at 127.31.237.190:43225
I20260808 10:48:44.748147 28740 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:48:44.748755 28740 heartbeater.cc:511] Master 127.31.237.190:43225 requested a full tablet report, sending...
I20260808 10:48:44.749998 28257 ts_manager.cc:194] Registered new tserver with Master: 3d5552b276fb4a70899c11237323f8b6 (127.31.237.130:33551)
I20260808 10:48:44.750726 28257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:38563
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:48:44.770275 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:43225
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:45751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:44.886931 28747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:44.887184 28747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:44.887228 28747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:44.891108 28747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:44.891243 28747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:44.891369 28747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:48:44.896647 28747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:45751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:43225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:44.897773 28747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:44.898905 28747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:44.908948 28753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:44.909014 28758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:44.909024 28747 server_base.cc:1034] running on GCE node
W20260808 10:48:44.908972 28754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:44.909718 28747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:44.910452 28747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:44.911616 28747 hybrid_clock.cc:648] HybridClock initialized: now 1786186124911613 us; error 52 us; skew 500 ppm
Aug 08 10:48:44 dist-test-slave-twwt krb5kdc[28179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186124, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:44.914515 28747 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:48:44.915481 28747 webserver.cc:460] Webserver started at http://127.31.237.131:41987/ using document root <none> and password file <none>
I20260808 10:48:44.916232 28747 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:44.916338 28747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:44.916586 28747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:44.919269 28747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "0a8c9e2304f74c039766b64721987254"
format_stamp: "Formatted at 2026-08-08 10:48:44 on dist-test-slave-twwt"
server_key: "b8bcaa3612316f8244c7182be46c9ca1"
server_key_iv: "1b62e3444fb75f2c8d64ef73009dcd16"
server_key_version: "encryptionkey@0"
I20260808 10:48:44.919847 28747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "0a8c9e2304f74c039766b64721987254"
format_stamp: "Formatted at 2026-08-08 10:48:44 on dist-test-slave-twwt"
server_key: "b8bcaa3612316f8244c7182be46c9ca1"
server_key_iv: "1b62e3444fb75f2c8d64ef73009dcd16"
server_key_version: "encryptionkey@0"
I20260808 10:48:44.923595 28747 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:48:44.925976 28767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:44.927233 28747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:48:44.927371 28747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "0a8c9e2304f74c039766b64721987254"
format_stamp: "Formatted at 2026-08-08 10:48:44 on dist-test-slave-twwt"
server_key: "b8bcaa3612316f8244c7182be46c9ca1"
server_key_iv: "1b62e3444fb75f2c8d64ef73009dcd16"
server_key_version: "encryptionkey@0"
I20260808 10:48:44.927474 28747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:44.951814 28747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:44.954732 28747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:44.954941 28747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:44.955569 28747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:44.956419 28747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:44.956467 28747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:44.956537 28747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:44.956574 28747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:44.965835 28747 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:43905
I20260808 10:48:44.965874 28942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:43905 every 8 connection(s)
I20260808 10:48:44.966814 28747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:48:44.968185 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 28747
I20260808 10:48:44.968376 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:48:44.968730 32694 external_mini_cluster.cc:1428] Setting key 9296801c381b45a86eed3201ce46b68b
Aug 08 10:48:44 dist-test-slave-twwt krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186124, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:48:44.981467 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:44.968940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:45281 (local address 127.31.237.190:43225)
0808 10:48:44.969136 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:44.969141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:44.970321 (+ 1180us) server_negotiation.cc:407] Connection header received
0808 10:48:44.971990 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:44.971993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:44.972053 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:44.972125 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:44.973342 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.973877 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:44.975114 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.975341 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:44.975429 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:44.977885 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:44.977888 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:44.977891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:44.977916 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:44.979486 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:44.979998 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:44.980001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:44.980003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:44.980088 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:44.980427 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:44.980431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:44.980432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:44.980625 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:44.980717 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:44.981220 (+ 503us) server_negotiation.cc:299] Negotiation successful
0808 10:48:44.981329 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:48:44.982535 28945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:44.969550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:43225 (local address 127.31.237.131:45281)
0808 10:48:44.970133 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:44.970207 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:44.971771 (+ 1564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:44.972299 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:44.972305 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:44.972674 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:44.973204 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:44.973217 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.973988 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:44.973990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:44.974998 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:44.975008 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.975478 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:44.975481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:44.975625 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:44.976194 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:44.976210 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:44.977723 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:44.979587 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:44.979592 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:44.979604 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:44.979853 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:44.980185 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:44.980188 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:44.980189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:44.980319 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:44.980742 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:44.980749 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:44.981008 (+ 259us) client_negotiation.cc:769] Sending connection context
0808 10:48:44.981169 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 10:48:44.981469 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":144,"threads_started":1}
I20260808 10:48:44.983839 28943 heartbeater.cc:347] Connected to a master server at 127.31.237.190:43225
I20260808 10:48:44.984117 28943 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:44.984633 28943 heartbeater.cc:511] Master 127.31.237.190:43225 requested a full tablet report, sending...
I20260808 10:48:44.985694 28257 ts_manager.cc:194] Registered new tserver with Master: 0a8c9e2304f74c039766b64721987254 (127.31.237.131:43905)
I20260808 10:48:44.986310 28257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:45281
I20260808 10:48:44.995385 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:48:45.006179 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:44.997484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42798 (local address 127.31.237.190:43225)
0808 10:48:44.997648 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:44.997653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:44.997795 (+ 142us) server_negotiation.cc:407] Connection header received
0808 10:48:44.997969 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:44.997972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:44.998022 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:44.998103 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:44.998896 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:44.999414 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.000276 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.000510 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.000617 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.002428 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:45.002434 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:45.002437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:45.002470 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:45.004044 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:45.004665 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:45.004673 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:45.004675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:45.004777 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:45.005146 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:45.005150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:45.005152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:45.005391 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:45.005569 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:45.005877 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 10:48:45.005997 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 10:48:45.009485 28257 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:48:45.013043 28257 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:48:45.028198 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.023003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34918 (local address 127.31.237.129:45519)
0808 10:48:45.023799 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:45.023806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:45.023827 (+ 21us) server_negotiation.cc:407] Connection header received
0808 10:48:45.023892 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:45.023896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:45.024051 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:45.024183 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:45.024435 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.025194 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.026780 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.027515 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.027660 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.027817 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:45.027944 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 10:48:45.028037 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":73,"threads_started":1}
I20260808 10:48:45.028370 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.022015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:43905 (local address 127.0.0.1:56584)
0808 10:48:45.023029 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:45.023082 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:45.023220 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:45.023861 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:45.023868 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:45.023890 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:45.024020 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:45.024025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.025687 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.025690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:45.026843 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:45.026850 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.027956 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.027960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:45.028058 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.028095 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 10:48:45.028142 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 10:48:45.028211 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":33,"thread_start_us":85,"threads_started":1}
I20260808 10:48:45.028187 28966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.022806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:45519 (local address 127.0.0.1:34918)
0808 10:48:45.023472 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:45.023487 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:45.023568 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:45.024184 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:45.024187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:45.024204 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:45.024304 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:45.024313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.025317 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.025322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:45.026667 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:45.026678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.027645 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.027649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:45.027755 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.027793 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 10:48:45.027905 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 10:48:45.028037 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":106,"threads_started":1}
I20260808 10:48:45.028858 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.022133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56988 (local address 127.31.237.130:33551)
0808 10:48:45.022772 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:45.022786 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:45.023530 (+ 744us) server_negotiation.cc:407] Connection header received
0808 10:48:45.023742 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:45.023748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:45.023955 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:45.024094 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:45.024596 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.025685 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.027186 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.028017 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.028130 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.028254 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:45.028589 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 10:48:45.028703 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":114,"threads_started":1}
I20260808 10:48:45.029048 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.022334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56584 (local address 127.31.237.131:43905)
0808 10:48:45.023278 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:45.023293 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:45.023341 (+ 48us) server_negotiation.cc:407] Connection header received
0808 10:48:45.023433 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:45.023438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:45.023667 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:45.024188 (+ 521us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:45.024344 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.025537 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.026963 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.027789 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.028317 (+ 528us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.028736 (+ 419us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:45.028786 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 10:48:45.028901 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":153,"threads_started":1}
I20260808 10:48:45.029292 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.022026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:33551 (local address 127.0.0.1:56988)
0808 10:48:45.023433 (+ 1407us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:45.023446 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:45.023589 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:45.024307 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:45.024310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:45.024338 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:45.024462 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:45.024470 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.025857 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.025861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:45.027045 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:45.027056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.028316 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.028319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:45.028424 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.028445 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:48:45.029061 (+ 616us) client_negotiation.cc:240] Negotiation successful
0808 10:48:45.029152 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1337,"mutex_wait_us":21,"thread_start_us":148,"threads_started":1}
I20260808 10:48:45.031821 28434 tablet_service.cc:1467] Processing CreateTablet for tablet d9343ec3bcc64006a40e8515b3a8ec65 (DEFAULT_TABLE table=test-table [id=542730e660ca41ef99184a9430562dae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:45.031970 28650 tablet_service.cc:1467] Processing CreateTablet for tablet d9343ec3bcc64006a40e8515b3a8ec65 (DEFAULT_TABLE table=test-table [id=542730e660ca41ef99184a9430562dae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:45.031970 28839 tablet_service.cc:1467] Processing CreateTablet for tablet d9343ec3bcc64006a40e8515b3a8ec65 (DEFAULT_TABLE table=test-table [id=542730e660ca41ef99184a9430562dae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:45.032967 28434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9343ec3bcc64006a40e8515b3a8ec65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:45.033036 28839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9343ec3bcc64006a40e8515b3a8ec65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:45.033036 28650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9343ec3bcc64006a40e8515b3a8ec65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:45.039897 28974 tablet_bootstrap.cc:492] T d9343ec3bcc64006a40e8515b3a8ec65 P 803dd845e2f241dcb159d17217d788f8: Bootstrap starting.
I20260808 10:48:45.040918 28976 tablet_bootstrap.cc:492] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254: Bootstrap starting.
I20260808 10:48:45.041275 28977 tablet_bootstrap.cc:492] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6: Bootstrap starting.
I20260808 10:48:45.042305 28974 tablet_bootstrap.cc:654] T d9343ec3bcc64006a40e8515b3a8ec65 P 803dd845e2f241dcb159d17217d788f8: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:45.043236 28974 log.cc:824] T d9343ec3bcc64006a40e8515b3a8ec65 P 803dd845e2f241dcb159d17217d788f8: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:45.043282 28976 tablet_bootstrap.cc:654] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:45.043282 28977 tablet_bootstrap.cc:654] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:45.044674 28977 log.cc:824] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:45.044674 28976 log.cc:824] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:45.045719 28974 tablet_bootstrap.cc:492] T d9343ec3bcc64006a40e8515b3a8ec65 P 803dd845e2f241dcb159d17217d788f8: No bootstrap required, opened a new log
I20260808 10:48:45.045958 28974 ts_tablet_manager.cc:1397] T d9343ec3bcc64006a40e8515b3a8ec65 P 803dd845e2f241dcb159d17217d788f8: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:48:45.046741 28976 tablet_bootstrap.cc:492] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254: No bootstrap required, opened a new log
I20260808 10:48:45.046929 28976 ts_tablet_manager.cc:1397] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:48:45.047144 28977 tablet_bootstrap.cc:492] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6: No bootstrap required, opened a new log
I20260808 10:48:45.047322 28977 ts_tablet_manager.cc:1397] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:48:45.050832 28974 raft_consensus.cc:348] T d9343ec3bcc64006a40e8515b3a8ec65 P 803dd845e2f241dcb159d17217d788f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5552b276fb4a70899c11237323f8b6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33551 } } peers { permanent_uuid: "0a8c9e2304f74c039766b64721987254" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 43905 } } peers { permanent_uuid: "803dd845e2f241dcb159d17217d788f8" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45519 } }
I20260808 10:48:45.050868 28976 raft_consensus.cc:348] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5552b276fb4a70899c11237323f8b6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33551 } } peers { permanent_uuid: "0a8c9e2304f74c039766b64721987254" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 43905 } } peers { permanent_uuid: "803dd845e2f241dcb159d17217d788f8" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45519 } }
I20260808 10:48:45.051105 28974 raft_consensus.cc:374] T d9343ec3bcc64006a40e8515b3a8ec65 P 803dd845e2f241dcb159d17217d788f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:45.051128 28976 raft_consensus.cc:374] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:45.051179 28974 raft_consensus.cc:729] T d9343ec3bcc64006a40e8515b3a8ec65 P 803dd845e2f241dcb159d17217d788f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 803dd845e2f241dcb159d17217d788f8, State: Initialized, Role: FOLLOWER
I20260808 10:48:45.051028 28977 raft_consensus.cc:348] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5552b276fb4a70899c11237323f8b6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33551 } } peers { permanent_uuid: "0a8c9e2304f74c039766b64721987254" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 43905 } } peers { permanent_uuid: "803dd845e2f241dcb159d17217d788f8" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45519 } }
I20260808 10:48:45.051227 28976 raft_consensus.cc:729] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8c9e2304f74c039766b64721987254, State: Initialized, Role: FOLLOWER
I20260808 10:48:45.051242 28977 raft_consensus.cc:374] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:45.051298 28977 raft_consensus.cc:729] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d5552b276fb4a70899c11237323f8b6, State: Initialized, Role: FOLLOWER
I20260808 10:48:45.051692 28974 consensus_queue.cc:260] T d9343ec3bcc64006a40e8515b3a8ec65 P 803dd845e2f241dcb159d17217d788f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5552b276fb4a70899c11237323f8b6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33551 } } peers { permanent_uuid: "0a8c9e2304f74c039766b64721987254" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 43905 } } peers { permanent_uuid: "803dd845e2f241dcb159d17217d788f8" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45519 } }
I20260808 10:48:45.051743 28977 consensus_queue.cc:260] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5552b276fb4a70899c11237323f8b6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33551 } } peers { permanent_uuid: "0a8c9e2304f74c039766b64721987254" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 43905 } } peers { permanent_uuid: "803dd845e2f241dcb159d17217d788f8" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45519 } }
I20260808 10:48:45.051743 28976 consensus_queue.cc:260] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5552b276fb4a70899c11237323f8b6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33551 } } peers { permanent_uuid: "0a8c9e2304f74c039766b64721987254" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 43905 } } peers { permanent_uuid: "803dd845e2f241dcb159d17217d788f8" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45519 } }
I20260808 10:48:45.052393 28539 heartbeater.cc:503] Master 127.31.237.190:43225 was elected leader, sending a full tablet report...
I20260808 10:48:45.052846 28977 ts_tablet_manager.cc:1428] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 10:48:45.052846 28976 ts_tablet_manager.cc:1428] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:48:45.053220 28943 heartbeater.cc:503] Master 127.31.237.190:43225 was elected leader, sending a full tablet report...
I20260808 10:48:45.054009 28974 ts_tablet_manager.cc:1428] T d9343ec3bcc64006a40e8515b3a8ec65 P 803dd845e2f241dcb159d17217d788f8: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260808 10:48:45.054245 28740 heartbeater.cc:503] Master 127.31.237.190:43225 was elected leader, sending a full tablet report...
I20260808 10:48:45.190372 28984 raft_consensus.cc:482] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:48:45.190647 28984 raft_consensus.cc:504] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5552b276fb4a70899c11237323f8b6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33551 } } peers { permanent_uuid: "0a8c9e2304f74c039766b64721987254" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 43905 } } peers { permanent_uuid: "803dd845e2f241dcb159d17217d788f8" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45519 } }
I20260808 10:48:45.191952 28984 leader_election.cc:288] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a8c9e2304f74c039766b64721987254 (127.31.237.131:43905), 803dd845e2f241dcb159d17217d788f8 (127.31.237.129:45519)
I20260808 10:48:45.196382 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.192463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:45519 (local address 127.31.237.130:44441)
0808 10:48:45.192879 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:45.192894 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:45.193039 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:45.193302 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:45.193305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:45.193334 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:45.193452 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:45.193457 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.194187 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.194192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:45.195154 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:45.195165 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.195931 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.195935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:45.196037 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.196063 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 10:48:45.196132 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 10:48:45.196217 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":20,"spinlock_wait_cycles":4480,"thread_start_us":140,"threads_started":1}
I20260808 10:48:45.196805 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.192843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:44441 (local address 127.31.237.129:45519)
0808 10:48:45.193000 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:45.193004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:45.193018 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:48:45.193133 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:45.193136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:45.193184 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:45.193257 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:45.193570 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.194064 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.195282 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.195797 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.195928 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.195983 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:45.196606 (+ 623us) server_negotiation.cc:299] Negotiation successful
0808 10:48:45.196690 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 10:48:45.196909 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.192213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:43905 (local address 127.31.237.130:48763)
0808 10:48:45.192365 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:45.192383 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:45.192502 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:45.192975 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:45.192979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:45.193029 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:45.193161 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:45.193168 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.194237 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.194240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:45.195498 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:45.195508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.196466 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.196470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:45.196589 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.196616 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:48:45.196690 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 10:48:45.196755 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":33,"spinlock_wait_cycles":38016}
I20260808 10:48:45.197046 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.192399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:48763 (local address 127.31.237.131:43905)
0808 10:48:45.192547 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:45.192554 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:45.192572 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:48:45.192633 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:45.192636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:45.192692 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:45.192786 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:45.193304 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.194094 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.195639 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.196340 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.196466 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.196517 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:45.196838 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 10:48:45.196919 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 10:48:45.197463 28466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9343ec3bcc64006a40e8515b3a8ec65" candidate_uuid: "3d5552b276fb4a70899c11237323f8b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "803dd845e2f241dcb159d17217d788f8" is_pre_election: true
I20260808 10:48:45.197716 28466 raft_consensus.cc:2454] T d9343ec3bcc64006a40e8515b3a8ec65 P 803dd845e2f241dcb159d17217d788f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d5552b276fb4a70899c11237323f8b6 in term 0.
I20260808 10:48:45.198201 28582 leader_election.cc:302] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d5552b276fb4a70899c11237323f8b6, 803dd845e2f241dcb159d17217d788f8; no voters:
I20260808 10:48:45.198271 28870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9343ec3bcc64006a40e8515b3a8ec65" candidate_uuid: "3d5552b276fb4a70899c11237323f8b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a8c9e2304f74c039766b64721987254" is_pre_election: true
I20260808 10:48:45.198530 28984 raft_consensus.cc:2781] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:48:45.198550 28870 raft_consensus.cc:2454] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d5552b276fb4a70899c11237323f8b6 in term 0.
I20260808 10:48:45.198650 28984 raft_consensus.cc:482] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:48:45.198711 28984 raft_consensus.cc:3037] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:45.199898 28984 raft_consensus.cc:504] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5552b276fb4a70899c11237323f8b6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33551 } } peers { permanent_uuid: "0a8c9e2304f74c039766b64721987254" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 43905 } } peers { permanent_uuid: "803dd845e2f241dcb159d17217d788f8" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45519 } }
I20260808 10:48:45.200278 28984 leader_election.cc:288] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [CANDIDATE]: Term 1 election: Requested vote from peers 0a8c9e2304f74c039766b64721987254 (127.31.237.131:43905), 803dd845e2f241dcb159d17217d788f8 (127.31.237.129:45519)
I20260808 10:48:45.200623 28870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9343ec3bcc64006a40e8515b3a8ec65" candidate_uuid: "3d5552b276fb4a70899c11237323f8b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a8c9e2304f74c039766b64721987254"
I20260808 10:48:45.200719 28466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9343ec3bcc64006a40e8515b3a8ec65" candidate_uuid: "3d5552b276fb4a70899c11237323f8b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "803dd845e2f241dcb159d17217d788f8"
I20260808 10:48:45.200753 28870 raft_consensus.cc:3037] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:45.200846 28466 raft_consensus.cc:3037] T d9343ec3bcc64006a40e8515b3a8ec65 P 803dd845e2f241dcb159d17217d788f8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:45.202132 28466 raft_consensus.cc:2454] T d9343ec3bcc64006a40e8515b3a8ec65 P 803dd845e2f241dcb159d17217d788f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d5552b276fb4a70899c11237323f8b6 in term 1.
I20260808 10:48:45.202132 28870 raft_consensus.cc:2454] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d5552b276fb4a70899c11237323f8b6 in term 1.
I20260808 10:48:45.202474 28582 leader_election.cc:302] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a8c9e2304f74c039766b64721987254, 3d5552b276fb4a70899c11237323f8b6; no voters:
I20260808 10:48:45.202687 28984 raft_consensus.cc:2781] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:45.202910 28984 raft_consensus.cc:686] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [term 1 LEADER]: Becoming Leader. State: Replica: 3d5552b276fb4a70899c11237323f8b6, State: Running, Role: LEADER
I20260808 10:48:45.203279 28984 consensus_queue.cc:237] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5552b276fb4a70899c11237323f8b6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33551 } } peers { permanent_uuid: "0a8c9e2304f74c039766b64721987254" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 43905 } } peers { permanent_uuid: "803dd845e2f241dcb159d17217d788f8" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45519 } }
I20260808 10:48:45.206504 28257 catalog_manager.cc:5495] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d5552b276fb4a70899c11237323f8b6 (127.31.237.130). New cstate: current_term: 1 leader_uuid: "3d5552b276fb4a70899c11237323f8b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d5552b276fb4a70899c11237323f8b6" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 33551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a8c9e2304f74c039766b64721987254" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 43905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "803dd845e2f241dcb159d17217d788f8" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45519 } health_report { overall_health: UNKNOWN } } }
W20260808 10:48:45.217855 28944 tablet.cc:2369] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:48:45.224462 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.220971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57002 (local address 127.31.237.130:33551)
0808 10:48:45.221097 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:45.221101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:45.221137 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:48:45.221208 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:45.221210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:45.221265 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:45.221352 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:48:45.221573 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.222090 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.222982 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.223227 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.223310 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.223689 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:48:45.224117 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:48:45.224233 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:45.224313 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 10:48:45.224350 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:48:45.230798 28870 raft_consensus.cc:1261] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254 [term 1 FOLLOWER]: Refusing update from remote peer 3d5552b276fb4a70899c11237323f8b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:48:45.230792 28466 raft_consensus.cc:1261] T d9343ec3bcc64006a40e8515b3a8ec65 P 803dd845e2f241dcb159d17217d788f8 [term 1 FOLLOWER]: Refusing update from remote peer 3d5552b276fb4a70899c11237323f8b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260808 10:48:45.231021 28741 tablet.cc:2369] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:48:45.231590 28984 consensus_queue.cc:1035] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "803dd845e2f241dcb159d17217d788f8" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:48:45.231841 28988 consensus_queue.cc:1035] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a8c9e2304f74c039766b64721987254" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 43905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:48:45.250505 28257 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:48:45.250855 28257 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:48:45.253674 28257 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e271d7a16e94f33be99a2d131c1eeeb: Sending DeleteTablet for 3 replicas of tablet d9343ec3bcc64006a40e8515b3a8ec65
I20260808 10:48:45.254658 28650 tablet_service.cc:1514] Processing DeleteTablet for tablet d9343ec3bcc64006a40e8515b3a8ec65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:45 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:56988
I20260808 10:48:45.254689 28434 tablet_service.cc:1514] Processing DeleteTablet for tablet d9343ec3bcc64006a40e8515b3a8ec65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:45 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:34918
I20260808 10:48:45.254739 28839 tablet_service.cc:1514] Processing DeleteTablet for tablet d9343ec3bcc64006a40e8515b3a8ec65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:45 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:56584
I20260808 10:48:45.255355 29012 tablet_replica.cc:330] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6: stopping tablet replica
I20260808 10:48:45.255708 29012 raft_consensus.cc:2229] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:48:45.256146 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 28343
I20260808 10:48:45.256204 29012 raft_consensus.cc:2258] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:45.261916 29012 ts_tablet_manager.cc:1905] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:45.263763 29011 tablet_replica.cc:330] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254: stopping tablet replica
I20260808 10:48:45.264187 29011 raft_consensus.cc:2229] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:48:45.264600 29011 raft_consensus.cc:2258] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:45.265496 29012 ts_tablet_manager.cc:1918] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:45.265596 29012 log.cc:1196] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/d9343ec3bcc64006a40e8515b3a8ec65
I20260808 10:48:45.265937 29012 ts_tablet_manager.cc:1939] T d9343ec3bcc64006a40e8515b3a8ec65 P 3d5552b276fb4a70899c11237323f8b6: Deleting consensus metadata
I20260808 10:48:45.267604 28231 catalog_manager.cc:4841] TS 3d5552b276fb4a70899c11237323f8b6 (127.31.237.130:33551): tablet d9343ec3bcc64006a40e8515b3a8ec65 (table test-table [id=542730e660ca41ef99184a9430562dae]) successfully deleted
I20260808 10:48:45.267894 29011 ts_tablet_manager.cc:1905] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:45.271414 29011 ts_tablet_manager.cc:1918] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:45.271541 29011 log.cc:1196] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/d9343ec3bcc64006a40e8515b3a8ec65
I20260808 10:48:45.272001 29011 ts_tablet_manager.cc:1939] T d9343ec3bcc64006a40e8515b3a8ec65 P 0a8c9e2304f74c039766b64721987254: Deleting consensus metadata
I20260808 10:48:45.273523 28229 catalog_manager.cc:4841] TS 0a8c9e2304f74c039766b64721987254 (127.31.237.131:43905): tablet d9343ec3bcc64006a40e8515b3a8ec65 (table test-table [id=542730e660ca41ef99184a9430562dae]) successfully deleted
W20260808 10:48:45.276610 28229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:45519 (error 108)
I20260808 10:48:45.277554 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 28552
I20260808 10:48:45.277922 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.277486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:45519 (local address 127.0.0.1:34934)
0808 10:48:45.277714 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:45.277812 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:45519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260808 10:48:45.293991 28229 catalog_manager.cc:4584] TS 803dd845e2f241dcb159d17217d788f8 (127.31.237.129:45519): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d9343ec3bcc64006a40e8515b3a8ec65: Network error: Client connection negotiation failed: client connection to 127.31.237.129:45519: connect: Connection refused (error 111)
I20260808 10:48:45.301209 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 28747
I20260808 10:48:45.314252 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 28201
2026-08-08T10:48:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2724 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T10:48:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:48:45Z Disabled control of system clock
I20260808 10:48:45.354146 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:38899
--webserver_interface=127.31.237.190
--webserver_port=0
--builtin_ntp_servers=127.31.237.148:34257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:38899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:48:45.470647 29019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:45.470921 29019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:45.470956 29019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:45.474920 29019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:48:45.474974 29019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:45.474989 29019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:45.475006 29019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:48:45.475021 29019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:48:45.479712 29019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:34257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:38899
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:38899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:45.480726 29019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:45.482071 29019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:45.490748 29028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:45.490744 29025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:45.490744 29026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:45.491107 29019 server_base.cc:1034] running on GCE node
I20260808 10:48:45.491671 29019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:45.492704 29019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:45.493888 29019 hybrid_clock.cc:648] HybridClock initialized: now 1786186125493856 us; error 44 us; skew 500 ppm
I20260808 10:48:45.495880 29019 webserver.cc:460] Webserver started at http://127.31.237.190:42233/ using document root <none> and password file <none>
I20260808 10:48:45.496475 29019 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:45.496538 29019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:45.496779 29019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:45.498904 29019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "198d29238e6a4b78a53288b6b99f6688"
format_stamp: "Formatted at 2026-08-08 10:48:45 on dist-test-slave-twwt"
server_key: "db4d4dcbfdfe5135875c08073264620e"
server_key_iv: "cb4ea7cf7d0d9874f720f779260fdd2a"
server_key_version: "encryptionkey@0"
I20260808 10:48:45.499485 29019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "198d29238e6a4b78a53288b6b99f6688"
format_stamp: "Formatted at 2026-08-08 10:48:45 on dist-test-slave-twwt"
server_key: "db4d4dcbfdfe5135875c08073264620e"
server_key_iv: "cb4ea7cf7d0d9874f720f779260fdd2a"
server_key_version: "encryptionkey@0"
I20260808 10:48:45.503345 29019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:48:45.505733 29041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:45.507222 29019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:48:45.507376 29019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "198d29238e6a4b78a53288b6b99f6688"
format_stamp: "Formatted at 2026-08-08 10:48:45 on dist-test-slave-twwt"
server_key: "db4d4dcbfdfe5135875c08073264620e"
server_key_iv: "cb4ea7cf7d0d9874f720f779260fdd2a"
server_key_version: "encryptionkey@0"
I20260808 10:48:45.507495 29019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:45.525610 29019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:45.526396 29019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:45.526651 29019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:45.534170 29019 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:38899
I20260808 10:48:45.534188 29127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:38899 every 8 connection(s)
I20260808 10:48:45.535257 29019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:48:45.538507 29128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:45.540717 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 29019
I20260808 10:48:45.540844 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:48:45.541174 32694 external_mini_cluster.cc:1428] Setting key f16767e1d7d47b1fad76222d184e4824
I20260808 10:48:45.545889 29128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688: Bootstrap starting.
I20260808 10:48:45.548588 29128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:45.549481 29128 log.cc:824] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:45.551054 29132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.542571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36284 (local address 127.31.237.190:38899)
0808 10:48:45.543243 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:45.543259 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:45.543295 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:48:45.543925 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:45.543942 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:45.544322 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:45.544642 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:45.545133 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.546395 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.547629 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.548125 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.548261 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.549171 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:45.549176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:45.549189 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:45.549588 (+ 399us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:45.549644 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:45.549664 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:45.549746 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:45.549945 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 10:48:45.550256 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":125,"threads_started":1}
I20260808 10:48:45.551762 29128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688: No bootstrap required, opened a new log
I20260808 10:48:45.555564 29128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198d29238e6a4b78a53288b6b99f6688" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38899 } }
I20260808 10:48:45.555783 29128 raft_consensus.cc:374] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:45.555816 29128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 198d29238e6a4b78a53288b6b99f6688, State: Initialized, Role: FOLLOWER
I20260808 10:48:45.556181 29128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198d29238e6a4b78a53288b6b99f6688" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38899 } }
I20260808 10:48:45.556291 29128 raft_consensus.cc:388] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:48:45.556335 29128 raft_consensus.cc:482] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:48:45.556409 29128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:45.557323 29128 raft_consensus.cc:504] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198d29238e6a4b78a53288b6b99f6688" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38899 } }
I20260808 10:48:45.557672 29128 leader_election.cc:302] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 198d29238e6a4b78a53288b6b99f6688; no voters:
I20260808 10:48:45.558037 29128 leader_election.cc:288] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:48:45.558136 29134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:45.558351 29134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [term 1 LEADER]: Becoming Leader. State: Replica: 198d29238e6a4b78a53288b6b99f6688, State: Running, Role: LEADER
I20260808 10:48:45.558765 29134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198d29238e6a4b78a53288b6b99f6688" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38899 } }
I20260808 10:48:45.559362 29128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:48:45.560551 29138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 198d29238e6a4b78a53288b6b99f6688. Latest consensus state: current_term: 1 leader_uuid: "198d29238e6a4b78a53288b6b99f6688" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198d29238e6a4b78a53288b6b99f6688" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38899 } } }
I20260808 10:48:45.560555 29136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "198d29238e6a4b78a53288b6b99f6688" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198d29238e6a4b78a53288b6b99f6688" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38899 } } }
I20260808 10:48:45.560688 29138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:45.560696 29136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688 [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:45.561105 29148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:48:45.564283 29148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:48:45.570030 29148 catalog_manager.cc:1347] Generated new cluster ID: a2e185b91d164c59a63887a495669159
I20260808 10:48:45.570101 29148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:48:45.602653 29148 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:48:45.604096 29148 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:48:45.613816 29148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688: Generated new TSK 0
I20260808 10:48:45.614724 29148 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:48:45.617925 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:38899
--builtin_ntp_servers=127.31.237.148:34257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:48:45.738498 29167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:45.738795 29167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:45.738833 29167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:45.742816 29167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:45.742887 29167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:45.742944 29167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:48:45.747972 29167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:34257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:38899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:45.749109 29167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:45.750533 29167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:45.760661 29178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:45.760712 29181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:45.760777 29179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:45.760845 29167 server_base.cc:1034] running on GCE node
I20260808 10:48:45.761451 29167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:45.762181 29167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:45.763381 29167 hybrid_clock.cc:648] HybridClock initialized: now 1786186125763362 us; error 31 us; skew 500 ppm
I20260808 10:48:45.765388 29167 webserver.cc:460] Webserver started at http://127.31.237.129:46501/ using document root <none> and password file <none>
I20260808 10:48:45.766026 29167 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:45.766122 29167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:45.766363 29167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:45.768623 29167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "272e2ed0b7914818a8b52dde25e84896"
format_stamp: "Formatted at 2026-08-08 10:48:45 on dist-test-slave-twwt"
server_key: "fd0e2011d0239f42fc282d3a6247dfe7"
server_key_iv: "88ffbf732287dba7cf3accdcd7a75f5c"
server_key_version: "encryptionkey@0"
I20260808 10:48:45.769243 29167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "272e2ed0b7914818a8b52dde25e84896"
format_stamp: "Formatted at 2026-08-08 10:48:45 on dist-test-slave-twwt"
server_key: "fd0e2011d0239f42fc282d3a6247dfe7"
server_key_iv: "88ffbf732287dba7cf3accdcd7a75f5c"
server_key_version: "encryptionkey@0"
I20260808 10:48:45.773216 29167 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 10:48:45.776106 29187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:45.777432 29167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:48:45.777577 29167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "272e2ed0b7914818a8b52dde25e84896"
format_stamp: "Formatted at 2026-08-08 10:48:45 on dist-test-slave-twwt"
server_key: "fd0e2011d0239f42fc282d3a6247dfe7"
server_key_iv: "88ffbf732287dba7cf3accdcd7a75f5c"
server_key_version: "encryptionkey@0"
I20260808 10:48:45.777701 29167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:45.799604 29167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:45.800439 29167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:45.800664 29167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:45.801337 29167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:45.802405 29167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:45.802484 29167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:45.802568 29167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:45.802613 29167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:45.812139 29167 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:45197
I20260808 10:48:45.812161 29368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:45197 every 8 connection(s)
I20260808 10:48:45.813148 29167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:48:45.815896 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 29167
I20260808 10:48:45.815999 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:48:45.816253 32694 external_mini_cluster.cc:1428] Setting key d7240a3bfa09b568d6020710486df5cd
I20260808 10:48:45.818439 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:38899
--builtin_ntp_servers=127.31.237.148:34257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:48:45.824324 29132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.815191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:56769 (local address 127.31.237.190:38899)
0808 10:48:45.815333 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:45.815336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:45.816534 (+ 1198us) server_negotiation.cc:407] Connection header received
0808 10:48:45.818189 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:45.818193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:45.818263 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:45.818377 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:45.819425 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.820277 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.822048 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.822268 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.822346 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.823774 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:45.823777 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:45.823779 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:45.823793 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:45.823837 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:45.823852 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:45.823931 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:45.824139 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 10:48:45.824202 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:48:45.825735 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:45.815588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:38899 (local address 127.31.237.129:56769)
0808 10:48:45.816341 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:45.816396 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:45.817966 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:45.818571 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:45.818582 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:45.818737 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:45.819264 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:45.819278 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.820429 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.820435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:45.821923 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:45.821938 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:45.822436 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:45.822443 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:45.822710 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:45.823410 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:48:45.823414 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:45.823484 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:48:45.823594 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:45.823956 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:45.823967 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:45.823979 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 10:48:45.824134 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 10:48:45.824505 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":140,"threads_started":1}
I20260808 10:48:45.827199 29370 heartbeater.cc:347] Connected to a master server at 127.31.237.190:38899
I20260808 10:48:45.827469 29370 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:45.828061 29370 heartbeater.cc:511] Master 127.31.237.190:38899 requested a full tablet report, sending...
I20260808 10:48:45.829679 29067 ts_manager.cc:194] Registered new tserver with Master: 272e2ed0b7914818a8b52dde25e84896 (127.31.237.129:45197)
I20260808 10:48:45.830942 29067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.237.129:56769
W20260808 10:48:45.941447 29376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:45.941774 29376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:45.941854 29376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:45.945919 29376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:45.946007 29376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:45.946099 29376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:48:45.951117 29376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:34257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:38899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:45.952741 29376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:45.954803 29376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:45.965543 29387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:45.965533 29382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:45.965533 29384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:45.966032 29376 server_base.cc:1034] running on GCE node
I20260808 10:48:45.966449 29376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:45.967144 29376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:45.968375 29376 hybrid_clock.cc:648] HybridClock initialized: now 1786186125968355 us; error 31 us; skew 500 ppm
I20260808 10:48:45.970242 29376 webserver.cc:460] Webserver started at http://127.31.237.130:37361/ using document root <none> and password file <none>
I20260808 10:48:45.970918 29376 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:45.971012 29376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:45.971256 29376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:45.973361 29376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "b99a80e7892f40f696839f1d1708c87e"
format_stamp: "Formatted at 2026-08-08 10:48:45 on dist-test-slave-twwt"
server_key: "64bc854ebca04e39539fe3ab67e47e29"
server_key_iv: "fbd7cbdc3ee8c7e6b3e00493be897d65"
server_key_version: "encryptionkey@0"
I20260808 10:48:45.973932 29376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "b99a80e7892f40f696839f1d1708c87e"
format_stamp: "Formatted at 2026-08-08 10:48:45 on dist-test-slave-twwt"
server_key: "64bc854ebca04e39539fe3ab67e47e29"
server_key_iv: "fbd7cbdc3ee8c7e6b3e00493be897d65"
server_key_version: "encryptionkey@0"
I20260808 10:48:45.977711 29376 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 10:48:45.980239 29394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:45.981492 29376 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:48:45.981638 29376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "b99a80e7892f40f696839f1d1708c87e"
format_stamp: "Formatted at 2026-08-08 10:48:45 on dist-test-slave-twwt"
server_key: "64bc854ebca04e39539fe3ab67e47e29"
server_key_iv: "fbd7cbdc3ee8c7e6b3e00493be897d65"
server_key_version: "encryptionkey@0"
I20260808 10:48:45.981752 29376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:45.998855 29376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:45.999539 29376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:45.999737 29376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:46.000365 29376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:46.001345 29376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:46.001418 29376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:46.001492 29376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:46.001536 29376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:46.011087 29376 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:34139
I20260808 10:48:46.011152 29561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:34139 every 8 connection(s)
I20260808 10:48:46.012135 29376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:48:46.017174 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 29376
I20260808 10:48:46.017323 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:48:46.017624 32694 external_mini_cluster.cc:1428] Setting key 4e96af64968a641379b5c9814dce5403
I20260808 10:48:46.019863 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:38899
--builtin_ntp_servers=127.31.237.148:34257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:48:46.022727 29132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.014100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:37799 (local address 127.31.237.190:38899)
0808 10:48:46.014275 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:46.014278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:46.015530 (+ 1252us) server_negotiation.cc:407] Connection header received
0808 10:48:46.017137 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:46.017140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:46.017201 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:46.017314 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:46.018358 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.019062 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.020365 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.020596 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.020712 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.022025 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:46.022032 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:46.022037 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:46.022056 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:46.022084 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:46.022097 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:46.022186 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:46.022417 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 10:48:46.022499 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 10:48:46.023572 29567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.014512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:38899 (local address 127.31.237.130:37799)
0808 10:48:46.015367 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:46.015416 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:46.016908 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:46.017504 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:46.017511 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:46.017597 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:46.018192 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:46.018214 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.019218 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.019222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:46.020224 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:46.020236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.020771 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.020776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:46.020942 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.021660 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:48:46.021663 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:46.021728 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:48:46.021797 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:46.022239 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:46.022249 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:46.022254 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:48:46.022380 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 10:48:46.022694 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":163,"threads_started":1}
I20260808 10:48:46.024844 29563 heartbeater.cc:347] Connected to a master server at 127.31.237.190:38899
I20260808 10:48:46.025132 29563 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:46.025851 29563 heartbeater.cc:511] Master 127.31.237.190:38899 requested a full tablet report, sending...
I20260808 10:48:46.027098 29067 ts_manager.cc:194] Registered new tserver with Master: b99a80e7892f40f696839f1d1708c87e (127.31.237.130:34139)
I20260808 10:48:46.027685 29067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.237.130:37799
W20260808 10:48:46.139182 29568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:46.139418 29568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:46.139456 29568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:46.143362 29568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:46.143411 29568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:46.143462 29568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:48:46.148193 29568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:34257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:38899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:46.149185 29568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:46.150499 29568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:46.160391 29581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:46.160446 29576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:46.160391 29575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:46.160830 29568 server_base.cc:1034] running on GCE node
I20260808 10:48:46.161300 29568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:46.162002 29568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:46.163203 29568 hybrid_clock.cc:648] HybridClock initialized: now 1786186126163188 us; error 31 us; skew 500 ppm
I20260808 10:48:46.165200 29568 webserver.cc:460] Webserver started at http://127.31.237.131:41697/ using document root <none> and password file <none>
I20260808 10:48:46.165865 29568 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:46.165968 29568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:46.166230 29568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:46.168467 29568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "3b65e2877b2346e8aa40216fa35eda42"
format_stamp: "Formatted at 2026-08-08 10:48:46 on dist-test-slave-twwt"
server_key: "6a1e8dfb9c6d2ceec433daabd052de01"
server_key_iv: "9b62d4cbb677df6b237fec8ed23c1e24"
server_key_version: "encryptionkey@0"
I20260808 10:48:46.169060 29568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "3b65e2877b2346e8aa40216fa35eda42"
format_stamp: "Formatted at 2026-08-08 10:48:46 on dist-test-slave-twwt"
server_key: "6a1e8dfb9c6d2ceec433daabd052de01"
server_key_iv: "9b62d4cbb677df6b237fec8ed23c1e24"
server_key_version: "encryptionkey@0"
I20260808 10:48:46.173002 29568 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260808 10:48:46.175632 29596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:46.176944 29568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:48:46.177085 29568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "3b65e2877b2346e8aa40216fa35eda42"
format_stamp: "Formatted at 2026-08-08 10:48:46 on dist-test-slave-twwt"
server_key: "6a1e8dfb9c6d2ceec433daabd052de01"
server_key_iv: "9b62d4cbb677df6b237fec8ed23c1e24"
server_key_version: "encryptionkey@0"
I20260808 10:48:46.177203 29568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:46.201807 29568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:46.202559 29568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:46.202836 29568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:46.203485 29568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:46.204480 29568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:46.204552 29568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:46.204627 29568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:46.204669 29568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:46.214015 29568 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:40369
I20260808 10:48:46.214043 29770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:40369 every 8 connection(s)
I20260808 10:48:46.214969 29568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:48:46.218111 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 29568
I20260808 10:48:46.218238 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:48:46.218636 32694 external_mini_cluster.cc:1428] Setting key 4034a7d1b64706c4ee19f081fa78f42b
I20260808 10:48:46.225219 29132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.216837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:58267 (local address 127.31.237.190:38899)
0808 10:48:46.217016 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:46.217021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:46.218459 (+ 1438us) server_negotiation.cc:407] Connection header received
0808 10:48:46.220343 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:46.220346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:46.220390 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:46.220447 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:46.221513 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.222044 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.223305 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.223525 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.223608 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.224757 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:46.224760 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:46.224762 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:46.224775 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:46.224795 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:46.224803 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:46.224900 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:46.225044 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 10:48:46.225099 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:48:46.226127 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.217423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:38899 (local address 127.31.237.131:58267)
0808 10:48:46.218279 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:46.218339 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:46.220098 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:46.220631 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:46.220639 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:46.220778 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:46.221385 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:46.221400 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.222163 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.222165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:46.223181 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:46.223203 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.223666 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.223669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:46.223834 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.224476 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:48:46.224478 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:46.224523 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:48:46.224591 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:46.224907 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:46.224915 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:46.224920 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:48:46.225025 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 10:48:46.225258 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":113,"threads_started":1}
I20260808 10:48:46.227319 29772 heartbeater.cc:347] Connected to a master server at 127.31.237.190:38899
I20260808 10:48:46.227557 29772 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:46.228086 29772 heartbeater.cc:511] Master 127.31.237.190:38899 requested a full tablet report, sending...
I20260808 10:48:46.229122 29067 ts_manager.cc:194] Registered new tserver with Master: 3b65e2877b2346e8aa40216fa35eda42 (127.31.237.131:40369)
I20260808 10:48:46.229700 29067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.237.131:58267
I20260808 10:48:46.232828 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:48:46.238751 29132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.234930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36288 (local address 127.31.237.190:38899)
0808 10:48:46.235044 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:46.235048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:46.235284 (+ 236us) server_negotiation.cc:407] Connection header received
0808 10:48:46.235341 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:46.235343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:46.235385 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:46.235460 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:46.235698 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.236240 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.237102 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.237315 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.237411 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.238269 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:46.238272 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:46.238274 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:46.238285 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:46.238300 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:46.238307 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:46.238418 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:46.238551 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 10:48:46.238584 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:48:46.241798 29067 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:48:46.245245 29067 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:48:46.260398 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.254070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:40369 (local address 127.0.0.1:44004)
0808 10:48:46.255685 (+ 1615us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:46.255700 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:46.255800 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:46.256228 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:46.256232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:46.256256 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:46.256364 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:46.256370 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.257397 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.257402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:46.258843 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:46.258855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.259852 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.259855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:46.259960 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.259998 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 10:48:46.260103 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 10:48:46.260238 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":100,"thread_start_us":94,"threads_started":1}
I20260808 10:48:46.260511 29790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.254240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44004 (local address 127.31.237.131:40369)
0808 10:48:46.255649 (+ 1409us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:46.255662 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:46.255806 (+ 144us) server_negotiation.cc:407] Connection header received
0808 10:48:46.255919 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:46.255923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:46.256084 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:46.256244 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:46.256460 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.257271 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.258997 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.259715 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.259864 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.260135 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:46.260189 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 10:48:46.260341 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1300,"thread_start_us":145,"threads_started":1}
I20260808 10:48:46.260862 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.254235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:45197 (local address 127.0.0.1:43356)
0808 10:48:46.255158 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:46.255210 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:46.255364 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:46.255970 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:46.255976 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:46.255998 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:46.256150 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:46.256156 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.257604 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.257608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:46.259085 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:46.259093 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.260386 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.260390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:46.260496 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.260531 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 10:48:46.260655 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 10:48:46.260719 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":52,"thread_start_us":134,"threads_started":1}
I20260808 10:48:46.261502 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.254970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43356 (local address 127.31.237.129:45197)
0808 10:48:46.255469 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:46.255482 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:46.255509 (+ 27us) server_negotiation.cc:407] Connection header received
0808 10:48:46.255594 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:46.255601 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:46.255826 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:46.255988 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:46.256306 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.257474 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.259191 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.260000 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.260130 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.260306 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:46.261242 (+ 936us) server_negotiation.cc:299] Negotiation successful
0808 10:48:46.261360 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":131,"threads_started":1}
I20260808 10:48:46.263391 29793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.254961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52608 (local address 127.31.237.130:34139)
0808 10:48:46.255425 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:46.255444 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:46.255810 (+ 366us) server_negotiation.cc:407] Connection header received
0808 10:48:46.255897 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:46.255903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:46.256084 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:46.256244 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:46.256662 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.257744 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.259342 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.260165 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.260275 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.260427 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:46.263118 (+ 2691us) server_negotiation.cc:299] Negotiation successful
0808 10:48:46.263222 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":151,"threads_started":1}
I20260808 10:48:46.263662 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.254557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:34139 (local address 127.0.0.1:52608)
0808 10:48:46.255598 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:46.255610 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:46.255706 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:46.256414 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:46.256419 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:46.256443 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:46.256534 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:46.256540 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.257875 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.257888 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:46.259214 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:46.259225 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.262683 (+ 3458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.262688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:46.262777 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.262802 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 10:48:46.262882 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 10:48:46.263523 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":982,"mutex_wait_us":223,"thread_start_us":144,"threads_started":1}
I20260808 10:48:46.264183 29670 tablet_service.cc:1467] Processing CreateTablet for tablet 3b39d0b6ce4e419ea0a9bc0187855ee2 (DEFAULT_TABLE table=test-table [id=c79d9241c5ab404db3c52a7561b0940e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:46.264392 29270 tablet_service.cc:1467] Processing CreateTablet for tablet 3b39d0b6ce4e419ea0a9bc0187855ee2 (DEFAULT_TABLE table=test-table [id=c79d9241c5ab404db3c52a7561b0940e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:46.265358 29670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b39d0b6ce4e419ea0a9bc0187855ee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:46.265537 29270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b39d0b6ce4e419ea0a9bc0187855ee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:46.266798 29472 tablet_service.cc:1467] Processing CreateTablet for tablet 3b39d0b6ce4e419ea0a9bc0187855ee2 (DEFAULT_TABLE table=test-table [id=c79d9241c5ab404db3c52a7561b0940e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:46.267838 29472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b39d0b6ce4e419ea0a9bc0187855ee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:46.272099 29795 tablet_bootstrap.cc:492] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896: Bootstrap starting.
I20260808 10:48:46.273594 29796 tablet_bootstrap.cc:492] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42: Bootstrap starting.
I20260808 10:48:46.274308 29795 tablet_bootstrap.cc:654] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:46.274906 29797 tablet_bootstrap.cc:492] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e: Bootstrap starting.
I20260808 10:48:46.275278 29795 log.cc:824] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:46.275808 29796 tablet_bootstrap.cc:654] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:46.276680 29796 log.cc:824] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:46.277304 29797 tablet_bootstrap.cc:654] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:46.277575 29795 tablet_bootstrap.cc:492] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896: No bootstrap required, opened a new log
I20260808 10:48:46.277774 29795 ts_tablet_manager.cc:1397] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 10:48:46.278134 29797 log.cc:824] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:46.278609 29796 tablet_bootstrap.cc:492] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42: No bootstrap required, opened a new log
I20260808 10:48:46.278829 29796 ts_tablet_manager.cc:1397] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20260808 10:48:46.280196 29797 tablet_bootstrap.cc:492] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e: No bootstrap required, opened a new log
I20260808 10:48:46.280433 29797 ts_tablet_manager.cc:1397] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:48:46.282392 29795 raft_consensus.cc:348] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b65e2877b2346e8aa40216fa35eda42" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 40369 } } peers { permanent_uuid: "272e2ed0b7914818a8b52dde25e84896" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45197 } } peers { permanent_uuid: "b99a80e7892f40f696839f1d1708c87e" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34139 } }
I20260808 10:48:46.282437 29796 raft_consensus.cc:348] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b65e2877b2346e8aa40216fa35eda42" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 40369 } } peers { permanent_uuid: "272e2ed0b7914818a8b52dde25e84896" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45197 } } peers { permanent_uuid: "b99a80e7892f40f696839f1d1708c87e" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34139 } }
I20260808 10:48:46.282694 29796 raft_consensus.cc:374] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:46.282698 29795 raft_consensus.cc:374] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:46.282752 29795 raft_consensus.cc:729] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 272e2ed0b7914818a8b52dde25e84896, State: Initialized, Role: FOLLOWER
I20260808 10:48:46.282752 29796 raft_consensus.cc:729] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b65e2877b2346e8aa40216fa35eda42, State: Initialized, Role: FOLLOWER
I20260808 10:48:46.283213 29796 consensus_queue.cc:260] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b65e2877b2346e8aa40216fa35eda42" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 40369 } } peers { permanent_uuid: "272e2ed0b7914818a8b52dde25e84896" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45197 } } peers { permanent_uuid: "b99a80e7892f40f696839f1d1708c87e" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34139 } }
I20260808 10:48:46.283213 29795 consensus_queue.cc:260] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b65e2877b2346e8aa40216fa35eda42" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 40369 } } peers { permanent_uuid: "272e2ed0b7914818a8b52dde25e84896" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45197 } } peers { permanent_uuid: "b99a80e7892f40f696839f1d1708c87e" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34139 } }
I20260808 10:48:46.284154 29796 ts_tablet_manager.cc:1428] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260808 10:48:46.284214 29795 ts_tablet_manager.cc:1428] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:48:46.284519 29772 heartbeater.cc:503] Master 127.31.237.190:38899 was elected leader, sending a full tablet report...
I20260808 10:48:46.284564 29370 heartbeater.cc:503] Master 127.31.237.190:38899 was elected leader, sending a full tablet report...
I20260808 10:48:46.285285 29797 raft_consensus.cc:348] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b65e2877b2346e8aa40216fa35eda42" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 40369 } } peers { permanent_uuid: "272e2ed0b7914818a8b52dde25e84896" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45197 } } peers { permanent_uuid: "b99a80e7892f40f696839f1d1708c87e" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34139 } }
I20260808 10:48:46.285534 29797 raft_consensus.cc:374] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:46.285600 29797 raft_consensus.cc:729] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b99a80e7892f40f696839f1d1708c87e, State: Initialized, Role: FOLLOWER
I20260808 10:48:46.286026 29797 consensus_queue.cc:260] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b65e2877b2346e8aa40216fa35eda42" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 40369 } } peers { permanent_uuid: "272e2ed0b7914818a8b52dde25e84896" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45197 } } peers { permanent_uuid: "b99a80e7892f40f696839f1d1708c87e" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34139 } }
I20260808 10:48:46.286727 29563 heartbeater.cc:503] Master 127.31.237.190:38899 was elected leader, sending a full tablet report...
I20260808 10:48:46.288020 29797 ts_tablet_manager.cc:1428] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260808 10:48:46.300320 29806 raft_consensus.cc:482] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:48:46.300508 29806 raft_consensus.cc:504] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b65e2877b2346e8aa40216fa35eda42" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 40369 } } peers { permanent_uuid: "272e2ed0b7914818a8b52dde25e84896" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45197 } } peers { permanent_uuid: "b99a80e7892f40f696839f1d1708c87e" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34139 } }
I20260808 10:48:46.301640 29806 leader_election.cc:288] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b65e2877b2346e8aa40216fa35eda42 (127.31.237.131:40369), 272e2ed0b7914818a8b52dde25e84896 (127.31.237.129:45197)
I20260808 10:48:46.305552 29790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.302167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:51955 (local address 127.31.237.131:40369)
0808 10:48:46.302350 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:46.302357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:46.302374 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:48:46.302434 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:46.302437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:46.302485 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:46.302561 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:46.303029 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.303525 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.304593 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.304981 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.305109 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.305162 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:46.305345 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 10:48:46.305426 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 10:48:46.305552 29567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.301878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:40369 (local address 127.31.237.130:51955)
0808 10:48:46.302061 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:46.302078 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:46.302168 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:46.302752 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:46.302755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:46.302771 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:46.302868 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:46.302873 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.303645 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.303650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:46.304459 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:46.304468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.305104 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.305108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:46.305221 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.305239 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 10:48:46.305327 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:48:46.305415 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260808 10:48:46.306005 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.302105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:56301 (local address 127.31.237.129:45197)
0808 10:48:46.302239 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:46.302243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:46.302831 (+ 588us) server_negotiation.cc:407] Connection header received
0808 10:48:46.302976 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:46.302979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:46.303033 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:46.303116 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:46.303412 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.303956 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.304983 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.305381 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.305515 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.305561 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:46.305798 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 10:48:46.305878 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:48:46.306288 29703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b39d0b6ce4e419ea0a9bc0187855ee2" candidate_uuid: "b99a80e7892f40f696839f1d1708c87e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b65e2877b2346e8aa40216fa35eda42" is_pre_election: true
I20260808 10:48:46.306391 29807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.301884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:45197 (local address 127.31.237.130:56301)
0808 10:48:46.302722 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:46.302734 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:46.302861 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:46.303129 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:46.303132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:46.303149 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:46.303289 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:46.303294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.304073 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.304078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:46.304844 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:46.304853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.305513 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.305517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:46.305627 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.305658 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 10:48:46.306189 (+ 531us) client_negotiation.cc:240] Negotiation successful
0808 10:48:46.306270 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"mutex_wait_us":330,"thread_start_us":170,"threads_started":1}
I20260808 10:48:46.306667 29703 raft_consensus.cc:2454] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b99a80e7892f40f696839f1d1708c87e in term 0.
I20260808 10:48:46.307132 29297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b39d0b6ce4e419ea0a9bc0187855ee2" candidate_uuid: "b99a80e7892f40f696839f1d1708c87e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "272e2ed0b7914818a8b52dde25e84896" is_pre_election: true
I20260808 10:48:46.307250 29404 leader_election.cc:302] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b65e2877b2346e8aa40216fa35eda42, b99a80e7892f40f696839f1d1708c87e; no voters:
I20260808 10:48:46.307439 29297 raft_consensus.cc:2454] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b99a80e7892f40f696839f1d1708c87e in term 0.
I20260808 10:48:46.307570 29806 raft_consensus.cc:2781] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:48:46.307657 29806 raft_consensus.cc:482] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:48:46.307729 29806 raft_consensus.cc:3037] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:46.308826 29806 raft_consensus.cc:504] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b65e2877b2346e8aa40216fa35eda42" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 40369 } } peers { permanent_uuid: "272e2ed0b7914818a8b52dde25e84896" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45197 } } peers { permanent_uuid: "b99a80e7892f40f696839f1d1708c87e" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34139 } }
I20260808 10:48:46.309177 29806 leader_election.cc:288] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [CANDIDATE]: Term 1 election: Requested vote from peers 3b65e2877b2346e8aa40216fa35eda42 (127.31.237.131:40369), 272e2ed0b7914818a8b52dde25e84896 (127.31.237.129:45197)
I20260808 10:48:46.309527 29703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b39d0b6ce4e419ea0a9bc0187855ee2" candidate_uuid: "b99a80e7892f40f696839f1d1708c87e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b65e2877b2346e8aa40216fa35eda42"
I20260808 10:48:46.309592 29297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b39d0b6ce4e419ea0a9bc0187855ee2" candidate_uuid: "b99a80e7892f40f696839f1d1708c87e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "272e2ed0b7914818a8b52dde25e84896"
I20260808 10:48:46.309666 29703 raft_consensus.cc:3037] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:46.309743 29297 raft_consensus.cc:3037] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:46.310943 29703 raft_consensus.cc:2454] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b99a80e7892f40f696839f1d1708c87e in term 1.
I20260808 10:48:46.311019 29297 raft_consensus.cc:2454] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b99a80e7892f40f696839f1d1708c87e in term 1.
I20260808 10:48:46.311280 29404 leader_election.cc:302] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b65e2877b2346e8aa40216fa35eda42, b99a80e7892f40f696839f1d1708c87e; no voters:
I20260808 10:48:46.311483 29806 raft_consensus.cc:2781] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:46.311818 29806 raft_consensus.cc:686] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [term 1 LEADER]: Becoming Leader. State: Replica: b99a80e7892f40f696839f1d1708c87e, State: Running, Role: LEADER
I20260808 10:48:46.312178 29806 consensus_queue.cc:237] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b65e2877b2346e8aa40216fa35eda42" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 40369 } } peers { permanent_uuid: "272e2ed0b7914818a8b52dde25e84896" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45197 } } peers { permanent_uuid: "b99a80e7892f40f696839f1d1708c87e" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34139 } }
W20260808 10:48:46.314393 29371 tablet.cc:2369] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:48:46.315812 29064 catalog_manager.cc:5495] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e reported cstate change: term changed from 0 to 1, leader changed from <none> to b99a80e7892f40f696839f1d1708c87e (127.31.237.130). New cstate: current_term: 1 leader_uuid: "b99a80e7892f40f696839f1d1708c87e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b65e2877b2346e8aa40216fa35eda42" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 40369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "272e2ed0b7914818a8b52dde25e84896" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b99a80e7892f40f696839f1d1708c87e" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34139 } health_report { overall_health: HEALTHY } } }
I20260808 10:48:46.340132 29793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.336610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52614 (local address 127.31.237.130:34139)
0808 10:48:46.336744 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:46.336748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:46.336762 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:48:46.336853 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:46.336855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:46.336920 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:46.337015 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:48:46.337273 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.337768 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.338589 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:46.338892 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:46.338973 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:46.339255 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:48:46.339668 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:48:46.339777 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:46.339885 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 10:48:46.339942 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:48:46.346550 29703 raft_consensus.cc:1261] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42 [term 1 FOLLOWER]: Refusing update from remote peer b99a80e7892f40f696839f1d1708c87e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:48:46.346550 29297 raft_consensus.cc:1261] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896 [term 1 FOLLOWER]: Refusing update from remote peer b99a80e7892f40f696839f1d1708c87e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:48:46.347390 29806 consensus_queue.cc:1035] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b65e2877b2346e8aa40216fa35eda42" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 40369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:48:46.347641 29808 consensus_queue.cc:1035] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [LEADER]: Connected to new peer: Peer: permanent_uuid: "272e2ed0b7914818a8b52dde25e84896" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:48:46.354859 29813 mvcc.cc:205] Tried to move back new op lower bound from 7316218373507891200 to 7316218373377998848. Current Snapshot: MvccSnapshot[applied={T|T < 7316218373507891200}]
I20260808 10:48:46.356235 29816 mvcc.cc:205] Tried to move back new op lower bound from 7316218373507891200 to 7316218373377998848. Current Snapshot: MvccSnapshot[applied={T|T < 7316218373507891200}]
I20260808 10:48:46.359886 29814 mvcc.cc:205] Tried to move back new op lower bound from 7316218373507891200 to 7316218373377998848. Current Snapshot: MvccSnapshot[applied={T|T < 7316218373507891200}]
I20260808 10:48:46.364840 29062 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:48:46.365034 29062 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:48:46.367700 29062 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 198d29238e6a4b78a53288b6b99f6688: Sending DeleteTablet for 3 replicas of tablet 3b39d0b6ce4e419ea0a9bc0187855ee2
I20260808 10:48:46.368538 29270 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b39d0b6ce4e419ea0a9bc0187855ee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:46 UTC) from {username='slave'} at 127.0.0.1:43356
I20260808 10:48:46.368528 29670 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b39d0b6ce4e419ea0a9bc0187855ee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:46 UTC) from {username='slave'} at 127.0.0.1:44004
I20260808 10:48:46.369112 29827 tablet_replica.cc:330] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896: stopping tablet replica
I20260808 10:48:46.369529 29827 raft_consensus.cc:2229] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:48:46.369594 29472 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b39d0b6ce4e419ea0a9bc0187855ee2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:46 UTC) from {username='slave'} at 127.0.0.1:52608
I20260808 10:48:46.370028 29827 raft_consensus.cc:2258] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 272e2ed0b7914818a8b52dde25e84896 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:46.370065 29832 tablet_replica.cc:330] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e: stopping tablet replica
I20260808 10:48:46.370373 29832 raft_consensus.cc:2229] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:48:46.370527 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 29167
I20260808 10:48:46.373441 29830 tablet_replica.cc:330] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42: stopping tablet replica
I20260808 10:48:46.375111 29830 raft_consensus.cc:2229] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:48:46.375135 29832 raft_consensus.cc:2258] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:46.375510 29830 raft_consensus.cc:2258] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:46.377200 29830 ts_tablet_manager.cc:1905] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:46.378360 29832 ts_tablet_manager.cc:1905] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:46.381515 29832 ts_tablet_manager.cc:1918] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:46.381621 29832 log.cc:1196] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/3b39d0b6ce4e419ea0a9bc0187855ee2
I20260808 10:48:46.381464 29830 ts_tablet_manager.cc:1918] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:46.381896 29830 log.cc:1196] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/3b39d0b6ce4e419ea0a9bc0187855ee2
I20260808 10:48:46.381939 29832 ts_tablet_manager.cc:1939] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P b99a80e7892f40f696839f1d1708c87e: Deleting consensus metadata
I20260808 10:48:46.382401 29830 ts_tablet_manager.cc:1939] T 3b39d0b6ce4e419ea0a9bc0187855ee2 P 3b65e2877b2346e8aa40216fa35eda42: Deleting consensus metadata
I20260808 10:48:46.383288 29044 catalog_manager.cc:4841] TS b99a80e7892f40f696839f1d1708c87e (127.31.237.130:34139): tablet 3b39d0b6ce4e419ea0a9bc0187855ee2 (table test-table [id=c79d9241c5ab404db3c52a7561b0940e]) successfully deleted
I20260808 10:48:46.384155 29046 catalog_manager.cc:4841] TS 3b65e2877b2346e8aa40216fa35eda42 (127.31.237.131:40369): tablet 3b39d0b6ce4e419ea0a9bc0187855ee2 (table test-table [id=c79d9241c5ab404db3c52a7561b0940e]) successfully deleted
W20260808 10:48:46.393489 29045 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:45197 (error 108)
I20260808 10:48:46.394356 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 29376
I20260808 10:48:46.394759 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:46.394356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:45197 (local address 127.0.0.1:43366)
0808 10:48:46.394505 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:46.394584 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:45197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 10:48:46.398406 29045 catalog_manager.cc:4584] TS 272e2ed0b7914818a8b52dde25e84896 (127.31.237.129:45197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b39d0b6ce4e419ea0a9bc0187855ee2: Network error: Client connection negotiation failed: client connection to 127.31.237.129:45197: connect: Connection refused (error 111)
I20260808 10:48:46.409013 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 29568
I20260808 10:48:46.419694 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 29019
2026-08-08T10:48:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1103 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:48:46 dist-test-slave-twwt krb5kdc[29838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:48:46 dist-test-slave-twwt krb5kdc[29838](info): set up 2 sockets
Aug 08 10:48:46 dist-test-slave-twwt krb5kdc[29838](info): commencing operation
krb5kdc: starting...
W20260808 10:48:47.743659 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:48:47 dist-test-slave-twwt krb5kdc[29838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:48:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:48:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:48:47.878198 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:33583
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:33583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:47.999451 29856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:47.999742 29856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:47.999822 29856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:48.004109 29856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:48:48.004222 29856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:48.004268 29856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:48.004463 29856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:48:48.004551 29856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:48:48.010542 29856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:33583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:33583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:48.011745 29856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:48.012883 29856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:48.022121 29869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:48.022178 29866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:48.022305 29856 server_base.cc:1034] running on GCE node
W20260808 10:48:48.022212 29867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:48.023138 29856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:48.024346 29856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:48.025579 29856 hybrid_clock.cc:648] HybridClock initialized: now 1786186128025520 us; error 80 us; skew 500 ppm
Aug 08 10:48:48 dist-test-slave-twwt krb5kdc[29838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186128, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:48.028859 29856 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:48:48.029891 29856 webserver.cc:460] Webserver started at http://127.31.237.190:44409/ using document root <none> and password file <none>
I20260808 10:48:48.030534 29856 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:48.030655 29856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:48.030941 29856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:48.033245 29856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "13d7030cadf945f39798d4a44a2ed171"
format_stamp: "Formatted at 2026-08-08 10:48:48 on dist-test-slave-twwt"
server_key: "ff3945a6103096ab1e86829d271cb1ac"
server_key_iv: "6386d4a7d6cd5114318d2361b7960a87"
server_key_version: "encryptionkey@0"
I20260808 10:48:48.033912 29856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "13d7030cadf945f39798d4a44a2ed171"
format_stamp: "Formatted at 2026-08-08 10:48:48 on dist-test-slave-twwt"
server_key: "ff3945a6103096ab1e86829d271cb1ac"
server_key_iv: "6386d4a7d6cd5114318d2361b7960a87"
server_key_version: "encryptionkey@0"
I20260808 10:48:48.038206 29856 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260808 10:48:48.041010 29882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:48.042279 29856 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:48:48.042446 29856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "13d7030cadf945f39798d4a44a2ed171"
format_stamp: "Formatted at 2026-08-08 10:48:48 on dist-test-slave-twwt"
server_key: "ff3945a6103096ab1e86829d271cb1ac"
server_key_iv: "6386d4a7d6cd5114318d2361b7960a87"
server_key_version: "encryptionkey@0"
I20260808 10:48:48.042570 29856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:48.063206 29856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:48.066187 29856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:48.066397 29856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:48.073982 29856 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:33583
I20260808 10:48:48.074007 29963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:33583 every 8 connection(s)
I20260808 10:48:48.074975 29856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:48:48.075975 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 29856
I20260808 10:48:48.076139 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:48:48.076437 32694 external_mini_cluster.cc:1428] Setting key d5136f8c3a1abc8134aca8b70d369b86
I20260808 10:48:48.078171 29964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:48.083849 29964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171: Bootstrap starting.
Aug 08 10:48:48 dist-test-slave-twwt krb5kdc[29838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:48:48.086540 29964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:48.087494 29964 log.cc:824] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:48.089628 29964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171: No bootstrap required, opened a new log
I20260808 10:48:48.091329 29970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.077897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39666 (local address 127.31.237.190:33583)
0808 10:48:48.078388 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:48.078404 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:48.078447 (+ 43us) server_negotiation.cc:407] Connection header received
0808 10:48:48.079175 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:48.079194 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:48.079537 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:48.079888 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:48.080761 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.081772 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.082665 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.083154 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.083331 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.086180 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:48.086185 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:48.086198 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:48.086226 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:48.088383 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:48.088841 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:48.088854 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:48.088859 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:48.088972 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:48.089272 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:48.089276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:48.089277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:48.089707 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:48.089926 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:48.090250 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 10:48:48.090554 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":150,"threads_started":1}
I20260808 10:48:48.093731 29964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d7030cadf945f39798d4a44a2ed171" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33583 } }
I20260808 10:48:48.093993 29964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:48.094085 29964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13d7030cadf945f39798d4a44a2ed171, State: Initialized, Role: FOLLOWER
I20260808 10:48:48.094599 29964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d7030cadf945f39798d4a44a2ed171" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33583 } }
I20260808 10:48:48.094797 29964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:48:48.094863 29964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:48:48.094975 29964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:48.095901 29964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d7030cadf945f39798d4a44a2ed171" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33583 } }
I20260808 10:48:48.096274 29964 leader_election.cc:302] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13d7030cadf945f39798d4a44a2ed171; no voters:
I20260808 10:48:48.096650 29964 leader_election.cc:288] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:48:48.096899 29973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:48.097177 29973 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [term 1 LEADER]: Becoming Leader. State: Replica: 13d7030cadf945f39798d4a44a2ed171, State: Running, Role: LEADER
I20260808 10:48:48.097487 29973 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d7030cadf945f39798d4a44a2ed171" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33583 } }
I20260808 10:48:48.097982 29964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:48:48.099193 29978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13d7030cadf945f39798d4a44a2ed171. Latest consensus state: current_term: 1 leader_uuid: "13d7030cadf945f39798d4a44a2ed171" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d7030cadf945f39798d4a44a2ed171" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33583 } } }
I20260808 10:48:48.099330 29978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:48.099606 29975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13d7030cadf945f39798d4a44a2ed171" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d7030cadf945f39798d4a44a2ed171" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 33583 } } }
I20260808 10:48:48.099716 29975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171 [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:48.099741 29989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:48:48.102722 29989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:48:48.108758 29989 catalog_manager.cc:1347] Generated new cluster ID: 2822e0e6f0cd45999f8230c1a0f12978
I20260808 10:48:48.108845 29989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:48:48.118512 29989 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:48:48.119789 29989 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:48:48.126210 29989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171: Generated new TSK 0
I20260808 10:48:48.127048 29989 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.237.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
Entry for principal kudu/127.31.237.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab.
I20260808 10:48:48.159776 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33583
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:48.277612 30013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:48.277864 30013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:48.277905 30013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:48.281827 30013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:48.281884 30013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:48.281940 30013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:48:48.287161 30013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.129.keytab
--principal=kudu/127.31.237.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:48.288139 30013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:48.289109 30013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:48:48.298818 30013 server_base.cc:1034] running on GCE node
W20260808 10:48:48.298794 30023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:48.298794 30022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:48.298839 30027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:48.299533 30013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:48.300276 30013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:48.301496 30013 hybrid_clock.cc:648] HybridClock initialized: now 1786186128301463 us; error 43 us; skew 500 ppm
Aug 08 10:48:48 dist-test-slave-twwt krb5kdc[29838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186128, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:48.304378 30013 init.cc:377] Logged in from keytab as kudu/127.31.237.129@KRBTEST.COM (short username kudu)
I20260808 10:48:48.305272 30013 webserver.cc:460] Webserver started at http://127.31.237.129:34077/ using document root <none> and password file <none>
I20260808 10:48:48.305847 30013 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:48.305941 30013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:48.306183 30013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:48.308312 30013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "f075a46284e747eea28ef4aa9baf8f07"
format_stamp: "Formatted at 2026-08-08 10:48:48 on dist-test-slave-twwt"
server_key: "359e17e08713c1213d44250aac51f11d"
server_key_iv: "94cdd153f3059ded5164c392122ae39b"
server_key_version: "encryptionkey@0"
I20260808 10:48:48.308950 30013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "f075a46284e747eea28ef4aa9baf8f07"
format_stamp: "Formatted at 2026-08-08 10:48:48 on dist-test-slave-twwt"
server_key: "359e17e08713c1213d44250aac51f11d"
server_key_iv: "94cdd153f3059ded5164c392122ae39b"
server_key_version: "encryptionkey@0"
I20260808 10:48:48.312870 30013 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 10:48:48.315593 30038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:48.316954 30013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:48:48.317126 30013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "f075a46284e747eea28ef4aa9baf8f07"
format_stamp: "Formatted at 2026-08-08 10:48:48 on dist-test-slave-twwt"
server_key: "359e17e08713c1213d44250aac51f11d"
server_key_iv: "94cdd153f3059ded5164c392122ae39b"
server_key_version: "encryptionkey@0"
I20260808 10:48:48.317253 30013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:48.339205 30013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:48.342410 30013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:48.342654 30013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:48.343293 30013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:48.344491 30013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:48.344568 30013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:48.344641 30013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:48.344672 30013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:48.354189 30013 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:41255
I20260808 10:48:48.354219 30220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:41255 every 8 connection(s)
I20260808 10:48:48.355166 30013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:48:48.357637 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 30013
I20260808 10:48:48.357793 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:48:48.358156 32694 external_mini_cluster.cc:1428] Setting key 1fb43dcaad39eb0b176e0f20867bdb37
Aug 08 10:48:48 dist-test-slave-twwt krb5kdc[29838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186128, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.129@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.130@KRBTEST.COM; defaulting to no policy
I20260808 10:48:48.370009 29970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.357005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:41389 (local address 127.31.237.190:33583)
0808 10:48:48.357156 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:48.357160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:48.358163 (+ 1003us) server_negotiation.cc:407] Connection header received
0808 10:48:48.359611 (+ 1448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:48.359614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:48.359669 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:48.359770 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:48.360953 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.361533 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.362772 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.363201 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.363330 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.366077 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:48.366084 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:48.366089 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:48.366125 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:48.367829 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:48.368352 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:48.368357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:48.368359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:48.368450 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:48.368786 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:48.368791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:48.368794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:48.369082 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:48.369197 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:48.369734 (+ 537us) server_negotiation.cc:299] Negotiation successful
0808 10:48:48.369881 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 10:48:48.370946 30226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.357442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33583 (local address 127.31.237.129:41389)
0808 10:48:48.358004 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:48.358048 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:48.359389 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:48.359952 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:48.359960 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:48.360306 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:48.360794 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.360807 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.361708 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.361711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.362595 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.362653 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.363357 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.363360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.363517 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.364137 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:48.364154 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:48.365891 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:48.367960 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:48.367965 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:48.367977 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:48.368240 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:48.368550 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:48.368553 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:48.368554 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:48.368686 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:48.369189 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:48.369195 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:48.369429 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 10:48:48.369649 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 10:48:48.369915 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":156,"threads_started":1}
I20260808 10:48:48.372370 30223 heartbeater.cc:347] Connected to a master server at 127.31.237.190:33583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.130@KRBTEST.COM" created.
I20260808 10:48:48.372637 30223 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:48.373180 30223 heartbeater.cc:511] Master 127.31.237.190:33583 requested a full tablet report, sending...
I20260808 10:48:48.375063 29910 ts_manager.cc:194] Registered new tserver with Master: f075a46284e747eea28ef4aa9baf8f07 (127.31.237.129:41255)
I20260808 10:48:48.376220 29910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.129@KRBTEST.COM'} at 127.31.237.129:41389
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
Entry for principal kudu/127.31.237.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab.
I20260808 10:48:48.390035 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33583
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:48.508095 30230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:48.508455 30230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:48.508534 30230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:48.512614 30230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:48.512732 30230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:48.512825 30230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:48:48.518432 30230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.130.keytab
--principal=kudu/127.31.237.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:48.519657 30230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:48.520813 30230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:48.530805 30237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:48.530817 30240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:48.530880 30236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:48.531219 30230 server_base.cc:1034] running on GCE node
I20260808 10:48:48.531641 30230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:48.532327 30230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:48.533495 30230 hybrid_clock.cc:648] HybridClock initialized: now 1786186128533478 us; error 28 us; skew 500 ppm
Aug 08 10:48:48 dist-test-slave-twwt krb5kdc[29838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186128, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:48.536304 30230 init.cc:377] Logged in from keytab as kudu/127.31.237.130@KRBTEST.COM (short username kudu)
I20260808 10:48:48.537324 30230 webserver.cc:460] Webserver started at http://127.31.237.130:41645/ using document root <none> and password file <none>
I20260808 10:48:48.537890 30230 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:48.537950 30230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:48.538189 30230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:48.540298 30230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "05a323a0b51146a4bdc14d24087fa272"
format_stamp: "Formatted at 2026-08-08 10:48:48 on dist-test-slave-twwt"
server_key: "6572e6a18d622a26887df680d87be0e2"
server_key_iv: "26d518a9a79fe64c7918f5257ce54823"
server_key_version: "encryptionkey@0"
I20260808 10:48:48.540977 30230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "05a323a0b51146a4bdc14d24087fa272"
format_stamp: "Formatted at 2026-08-08 10:48:48 on dist-test-slave-twwt"
server_key: "6572e6a18d622a26887df680d87be0e2"
server_key_iv: "26d518a9a79fe64c7918f5257ce54823"
server_key_version: "encryptionkey@0"
I20260808 10:48:48.544679 30230 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 10:48:48.547255 30251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:48.548549 30230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:48:48.548671 30230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "05a323a0b51146a4bdc14d24087fa272"
format_stamp: "Formatted at 2026-08-08 10:48:48 on dist-test-slave-twwt"
server_key: "6572e6a18d622a26887df680d87be0e2"
server_key_iv: "26d518a9a79fe64c7918f5257ce54823"
server_key_version: "encryptionkey@0"
I20260808 10:48:48.548779 30230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:48.569772 30230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:48.573132 30230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:48.573366 30230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:48.574036 30230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:48.574954 30230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:48.575002 30230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:48.575074 30230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:48.575116 30230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:48.584674 30230 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:34601
I20260808 10:48:48.584702 30420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:34601 every 8 connection(s)
I20260808 10:48:48.585669 30230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:48:48.587949 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 30230
I20260808 10:48:48.588128 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:48:48.588483 32694 external_mini_cluster.cc:1428] Setting key 4f58cc8ba748000ca257dcaaf251cac8
Aug 08 10:48:48 dist-test-slave-twwt krb5kdc[29838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186128, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.130@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.237.131@KRBTEST.COM; defaulting to no policy
I20260808 10:48:48.602347 29970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.587723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:42097 (local address 127.31.237.190:33583)
0808 10:48:48.587932 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:48.587951 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:48.589026 (+ 1075us) server_negotiation.cc:407] Connection header received
0808 10:48:48.590752 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:48.590757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:48.590854 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:48.590944 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:48.592772 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.593317 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.594805 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.595054 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.595157 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.598378 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:48.598382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:48.598384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:48.598412 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:48.600193 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:48.600738 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:48.600741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:48.600743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:48.600829 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:48.601189 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:48.601193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:48.601195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:48.601390 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:48.601501 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:48.602046 (+ 545us) server_negotiation.cc:299] Negotiation successful
0808 10:48:48.602177 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 10:48:48.603515 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.588153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33583 (local address 127.31.237.130:42097)
0808 10:48:48.588824 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:48.588890 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:48.590499 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:48.591181 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:48.591194 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:48.591752 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:48.592545 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.592574 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.593462 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.593465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.594657 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.594673 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.595265 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.595272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.595534 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.596259 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:48.596289 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:48.598184 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:48.600320 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:48.600327 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:48.600345 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:48.600630 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:48.600934 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:48.600938 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:48.600940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:48.601091 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:48.601503 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:48.601512 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:48.601789 (+ 277us) client_negotiation.cc:769] Sending connection context
0808 10:48:48.601990 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 10:48:48.602349 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":134,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.131@KRBTEST.COM" created.
I20260808 10:48:48.604806 30421 heartbeater.cc:347] Connected to a master server at 127.31.237.190:33583
I20260808 10:48:48.605098 30421 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:48.605758 30421 heartbeater.cc:511] Master 127.31.237.190:33583 requested a full tablet report, sending...
I20260808 10:48:48.607126 29910 ts_manager.cc:194] Registered new tserver with Master: 05a323a0b51146a4bdc14d24087fa272 (127.31.237.130:34601)
I20260808 10:48:48.607862 29910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.130@KRBTEST.COM'} at 127.31.237.130:42097
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
Entry for principal kudu/127.31.237.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab.
I20260808 10:48:48.621728 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33583
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:42117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:48.741003 30427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:48.741271 30427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:48.741314 30427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:48.745285 30427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:48.745345 30427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:48.745404 30427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:48:48.750876 30427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:42117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.131.keytab
--principal=kudu/127.31.237.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:33583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:48.751973 30427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:48.753078 30427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:48.763072 30435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:48.763072 30441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:48.763103 30437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:48.763485 30427 server_base.cc:1034] running on GCE node
I20260808 10:48:48.763962 30427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:48.764627 30427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:48.765821 30427 hybrid_clock.cc:648] HybridClock initialized: now 1786186128765795 us; error 42 us; skew 500 ppm
Aug 08 10:48:48 dist-test-slave-twwt krb5kdc[29838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186128, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:48.768730 30427 init.cc:377] Logged in from keytab as kudu/127.31.237.131@KRBTEST.COM (short username kudu)
I20260808 10:48:48.769670 30427 webserver.cc:460] Webserver started at http://127.31.237.131:42473/ using document root <none> and password file <none>
I20260808 10:48:48.770275 30427 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:48.770380 30427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:48.770684 30427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:48.772840 30427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "669badac559241e09211e20458ab4122"
format_stamp: "Formatted at 2026-08-08 10:48:48 on dist-test-slave-twwt"
server_key: "ad08ea6f3592dc4ffba7f35eb2a15fbf"
server_key_iv: "2bd073f1e76aae3a143067e17db3017b"
server_key_version: "encryptionkey@0"
I20260808 10:48:48.773480 30427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "669badac559241e09211e20458ab4122"
format_stamp: "Formatted at 2026-08-08 10:48:48 on dist-test-slave-twwt"
server_key: "ad08ea6f3592dc4ffba7f35eb2a15fbf"
server_key_iv: "2bd073f1e76aae3a143067e17db3017b"
server_key_version: "encryptionkey@0"
I20260808 10:48:48.777452 30427 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260808 10:48:48.780160 30452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:48.781461 30427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:48:48.781605 30427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "669badac559241e09211e20458ab4122"
format_stamp: "Formatted at 2026-08-08 10:48:48 on dist-test-slave-twwt"
server_key: "ad08ea6f3592dc4ffba7f35eb2a15fbf"
server_key_iv: "2bd073f1e76aae3a143067e17db3017b"
server_key_version: "encryptionkey@0"
I20260808 10:48:48.781728 30427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:48.809545 30427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:48.812668 30427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:48.812903 30427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:48.813549 30427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:48.814468 30427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:48.814543 30427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:48.814643 30427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:48.814702 30427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:48.823956 30427 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:35151
I20260808 10:48:48.823982 30618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:35151 every 8 connection(s)
I20260808 10:48:48.824877 30427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:48:48.829901 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 30427
I20260808 10:48:48.830066 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:48:48.830403 32694 external_mini_cluster.cc:1428] Setting key 8722c0451fb8f665d18dd974988b7595
Aug 08 10:48:48 dist-test-slave-twwt krb5kdc[29838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186128, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.131@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:48:48.839350 29970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.826752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:41431 (local address 127.31.237.190:33583)
0808 10:48:48.826896 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:48.826900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:48.827753 (+ 853us) server_negotiation.cc:407] Connection header received
0808 10:48:48.829105 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:48.829108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:48.829167 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:48.829274 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:48.830931 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.831694 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.833109 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.833329 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.833415 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.835936 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:48.835939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:48.835941 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:48.835967 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:48.837311 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:48.837834 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:48.837838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:48.837839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:48.837937 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:48.838299 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:48.838302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:48.838303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:48.838502 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:48.838584 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:48.839121 (+ 537us) server_negotiation.cc:299] Negotiation successful
0808 10:48:48.839227 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:48:48.840147 30622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.827142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:33583 (local address 127.31.237.131:41431)
0808 10:48:48.827599 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:48.827644 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:48.828883 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:48.829459 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:48.829466 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:48.829975 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:48.830725 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.830749 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.831855 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.831857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.832995 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.833006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.833467 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.833469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.833603 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.834213 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:48:48.834230 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:48.835776 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:48.837415 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:48.837420 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:48.837434 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:48.837690 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:48.838026 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:48:48.838028 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:48:48.838029 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:48:48.838170 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:48:48.838593 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:48.838598 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:48.838891 (+ 293us) client_negotiation.cc:769] Sending connection context
0808 10:48:48.839044 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 10:48:48.839297 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":105,"threads_started":1}
I20260808 10:48:48.841349 30619 heartbeater.cc:347] Connected to a master server at 127.31.237.190:33583
I20260808 10:48:48.841604 30619 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:48.842104 30619 heartbeater.cc:511] Master 127.31.237.190:33583 requested a full tablet report, sending...
I20260808 10:48:48.843279 29910 ts_manager.cc:194] Registered new tserver with Master: 669badac559241e09211e20458ab4122 (127.31.237.131:35151)
I20260808 10:48:48.844038 29910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.237.131@KRBTEST.COM'} at 127.31.237.131:41431
I20260808 10:48:48.844615 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:48:48.854187 29970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.846982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39668 (local address 127.31.237.190:33583)
0808 10:48:48.847095 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:48.847100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:48.847326 (+ 226us) server_negotiation.cc:407] Connection header received
0808 10:48:48.847417 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:48.847419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:48.847465 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:48.847531 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:48.848173 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.848667 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.849491 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.849693 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.849767 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.851168 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:48.851171 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:48.851173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:48.851196 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:48.852638 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:48.853078 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:48.853081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:48.853082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:48.853156 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:48.853448 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:48.853450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:48.853452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:48.853656 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:48.853758 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:48.853980 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 10:48:48.854072 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:48:48.857048 29910 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:48:48.860136 29910 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:48:48.875067 30634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.869018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:35151 (local address 127.0.0.1:56504)
0808 10:48:48.870311 (+ 1293us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:48.870322 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:48.870406 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:48.870885 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:48.870891 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:48.870913 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:48.871006 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.871012 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.872313 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.872316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.873466 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.873477 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.874461 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.874465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.874572 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.874643 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 10:48:48.874760 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 10:48:48.874889 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1234,"thread_start_us":109,"threads_started":1}
I20260808 10:48:48.875098 30641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.869879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56504 (local address 127.31.237.131:35151)
0808 10:48:48.870269 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:48.870280 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:48.870413 (+ 133us) server_negotiation.cc:407] Connection header received
0808 10:48:48.870511 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:48.870516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:48.870730 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:48.870884 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:48.871120 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.872113 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.873596 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.874328 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.874456 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.874593 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:48.874824 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 10:48:48.874933 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":166,"threads_started":1}
I20260808 10:48:48.875697 30633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.868806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:34601 (local address 127.0.0.1:43202)
0808 10:48:48.869299 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:48.869346 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:48.869498 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:48.870541 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:48.870548 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:48.870570 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:48.870760 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.870770 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.872373 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.872377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.873860 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.873872 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.875208 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.875212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.875311 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.875372 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 10:48:48.875482 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 10:48:48.875558 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":84,"thread_start_us":87,"threads_started":1}
I20260808 10:48:48.875869 30638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.869537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:41255 (local address 127.0.0.1:41774)
0808 10:48:48.870581 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:48.870593 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:48.870697 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:48.871357 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:48.871360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:48.871380 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:48.871477 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.871483 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.872882 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.872885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.874016 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.874024 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.875219 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.875225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.875384 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.875407 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 10:48:48.875658 (+ 251us) client_negotiation.cc:240] Negotiation successful
0808 10:48:48.875740 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":22,"thread_start_us":106,"threads_started":1}
I20260808 10:48:48.876194 30639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.869704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41774 (local address 127.31.237.129:41255)
0808 10:48:48.870886 (+ 1182us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:48.870900 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:48.870923 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:48:48.871012 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:48.871018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:48.871209 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:48.871366 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:48.871616 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.872737 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.874131 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.874909 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.875024 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.875147 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:48.875963 (+ 816us) server_negotiation.cc:299] Negotiation successful
0808 10:48:48.876061 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1072,"thread_start_us":98,"threads_started":1}
I20260808 10:48:48.876470 30636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.869020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43202 (local address 127.31.237.130:34601)
0808 10:48:48.869963 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:48.869976 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:48.870008 (+ 32us) server_negotiation.cc:407] Connection header received
0808 10:48:48.870104 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:48.870110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:48.870331 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:48.870530 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:48.870926 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.872059 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.873995 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.874890 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.875008 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.875131 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:48.875530 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 10:48:48.876306 (+ 776us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":815,"thread_start_us":143,"threads_started":1}
I20260808 10:48:48.878539 30525 tablet_service.cc:1467] Processing CreateTablet for tablet 26b1f915c6b04549884c3ef74c8a4bfd (DEFAULT_TABLE table=test-table [id=312049ecdf04433b804913c5259ef36a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:48.879216 30122 tablet_service.cc:1467] Processing CreateTablet for tablet 26b1f915c6b04549884c3ef74c8a4bfd (DEFAULT_TABLE table=test-table [id=312049ecdf04433b804913c5259ef36a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:48.879216 30316 tablet_service.cc:1467] Processing CreateTablet for tablet 26b1f915c6b04549884c3ef74c8a4bfd (DEFAULT_TABLE table=test-table [id=312049ecdf04433b804913c5259ef36a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:48.879732 30525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b1f915c6b04549884c3ef74c8a4bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:48.880280 30316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b1f915c6b04549884c3ef74c8a4bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:48.880280 30122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b1f915c6b04549884c3ef74c8a4bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:48.886705 30643 tablet_bootstrap.cc:492] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122: Bootstrap starting.
I20260808 10:48:48.888353 30645 tablet_bootstrap.cc:492] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07: Bootstrap starting.
I20260808 10:48:48.889043 30643 tablet_bootstrap.cc:654] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:48.890311 30645 tablet_bootstrap.cc:654] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:48.890340 30643 log.cc:824] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:48.891137 30645 log.cc:824] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:48.891568 30647 tablet_bootstrap.cc:492] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272: Bootstrap starting.
I20260808 10:48:48.892654 30643 tablet_bootstrap.cc:492] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122: No bootstrap required, opened a new log
I20260808 10:48:48.892874 30643 ts_tablet_manager.cc:1397] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:48:48.892961 30645 tablet_bootstrap.cc:492] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07: No bootstrap required, opened a new log
I20260808 10:48:48.893178 30645 ts_tablet_manager.cc:1397] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 10:48:48.893769 30647 tablet_bootstrap.cc:654] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:48.895071 30647 log.cc:824] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:48.896981 30645 raft_consensus.cc:348] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a323a0b51146a4bdc14d24087fa272" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34601 } } peers { permanent_uuid: "669badac559241e09211e20458ab4122" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35151 } } peers { permanent_uuid: "f075a46284e747eea28ef4aa9baf8f07" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 41255 } }
I20260808 10:48:48.897068 30643 raft_consensus.cc:348] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a323a0b51146a4bdc14d24087fa272" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34601 } } peers { permanent_uuid: "669badac559241e09211e20458ab4122" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35151 } } peers { permanent_uuid: "f075a46284e747eea28ef4aa9baf8f07" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 41255 } }
I20260808 10:48:48.897400 30647 tablet_bootstrap.cc:492] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272: No bootstrap required, opened a new log
I20260808 10:48:48.897392 30643 raft_consensus.cc:374] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:48.897423 30645 raft_consensus.cc:374] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:48.897457 30643 raft_consensus.cc:729] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 669badac559241e09211e20458ab4122, State: Initialized, Role: FOLLOWER
I20260808 10:48:48.897571 30645 raft_consensus.cc:729] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f075a46284e747eea28ef4aa9baf8f07, State: Initialized, Role: FOLLOWER
I20260808 10:48:48.897660 30647 ts_tablet_manager.cc:1397] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:48:48.897990 30643 consensus_queue.cc:260] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a323a0b51146a4bdc14d24087fa272" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34601 } } peers { permanent_uuid: "669badac559241e09211e20458ab4122" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35151 } } peers { permanent_uuid: "f075a46284e747eea28ef4aa9baf8f07" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 41255 } }
I20260808 10:48:48.898190 30645 consensus_queue.cc:260] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a323a0b51146a4bdc14d24087fa272" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34601 } } peers { permanent_uuid: "669badac559241e09211e20458ab4122" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35151 } } peers { permanent_uuid: "f075a46284e747eea28ef4aa9baf8f07" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 41255 } }
I20260808 10:48:48.898846 30619 heartbeater.cc:503] Master 127.31.237.190:33583 was elected leader, sending a full tablet report...
I20260808 10:48:48.899103 30643 ts_tablet_manager.cc:1428] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:48:48.899436 30645 ts_tablet_manager.cc:1428] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 10:48:48.899857 30223 heartbeater.cc:503] Master 127.31.237.190:33583 was elected leader, sending a full tablet report...
I20260808 10:48:48.902263 30647 raft_consensus.cc:348] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a323a0b51146a4bdc14d24087fa272" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34601 } } peers { permanent_uuid: "669badac559241e09211e20458ab4122" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35151 } } peers { permanent_uuid: "f075a46284e747eea28ef4aa9baf8f07" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 41255 } }
I20260808 10:48:48.902550 30647 raft_consensus.cc:374] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:48.902657 30647 raft_consensus.cc:729] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a323a0b51146a4bdc14d24087fa272, State: Initialized, Role: FOLLOWER
I20260808 10:48:48.903191 30647 consensus_queue.cc:260] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a323a0b51146a4bdc14d24087fa272" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34601 } } peers { permanent_uuid: "669badac559241e09211e20458ab4122" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35151 } } peers { permanent_uuid: "f075a46284e747eea28ef4aa9baf8f07" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 41255 } }
I20260808 10:48:48.903932 30421 heartbeater.cc:503] Master 127.31.237.190:33583 was elected leader, sending a full tablet report...
I20260808 10:48:48.904193 30647 ts_tablet_manager.cc:1428] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 10:48:48.907117 30659 raft_consensus.cc:482] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:48:48.907291 30659 raft_consensus.cc:504] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a323a0b51146a4bdc14d24087fa272" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34601 } } peers { permanent_uuid: "669badac559241e09211e20458ab4122" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35151 } } peers { permanent_uuid: "f075a46284e747eea28ef4aa9baf8f07" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 41255 } }
I20260808 10:48:48.908368 30659 leader_election.cc:288] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a323a0b51146a4bdc14d24087fa272 (127.31.237.130:34601), f075a46284e747eea28ef4aa9baf8f07 (127.31.237.129:41255)
I20260808 10:48:48.912366 30636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.908869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:41759 (local address 127.31.237.130:34601)
0808 10:48:48.908987 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:48.908992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:48.909008 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:48:48.909078 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:48.909081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:48.909138 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:48.909221 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:48.909509 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.910108 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.911275 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.911706 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.911800 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.911867 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:48.912177 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 10:48:48.912246 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:48:48.912384 30639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.908765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:53025 (local address 127.31.237.129:41255)
0808 10:48:48.908896 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:48.908904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:48.909210 (+ 306us) server_negotiation.cc:407] Connection header received
0808 10:48:48.909308 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:48.909311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:48.909375 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:48.909452 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:48.909746 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.910257 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.911347 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.911783 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.911902 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.911955 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:48.912198 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 10:48:48.912271 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:48:48.912372 30622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.908587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:34601 (local address 127.31.237.131:41759)
0808 10:48:48.908890 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:48.908907 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:48.908997 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:48.909243 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:48.909246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:48.909264 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:48.909395 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.909406 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.910213 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.910217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.911148 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.911159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.911835 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.911842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.912022 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.912070 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 10:48:48.912131 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:48:48.912226 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":21}
I20260808 10:48:48.912392 30664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.908623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:41255 (local address 127.31.237.131:53025)
0808 10:48:48.909092 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:48.909106 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:48.909216 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:48.909491 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:48.909494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:48.909513 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:48.909633 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.909640 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.910365 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.910367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.911201 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:48.911233 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.911913 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.911918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:48.912022 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.912077 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 10:48:48.912168 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 10:48:48.912240 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":27,"thread_start_us":117,"threads_started":1}
I20260808 10:48:48.913154 30346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b1f915c6b04549884c3ef74c8a4bfd" candidate_uuid: "669badac559241e09211e20458ab4122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a323a0b51146a4bdc14d24087fa272" is_pre_election: true
I20260808 10:48:48.913321 30150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b1f915c6b04549884c3ef74c8a4bfd" candidate_uuid: "669badac559241e09211e20458ab4122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f075a46284e747eea28ef4aa9baf8f07" is_pre_election: true
I20260808 10:48:48.913457 30346 raft_consensus.cc:2454] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 669badac559241e09211e20458ab4122 in term 0.
I20260808 10:48:48.913585 30150 raft_consensus.cc:2454] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 669badac559241e09211e20458ab4122 in term 0.
I20260808 10:48:48.913990 30453 leader_election.cc:302] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a323a0b51146a4bdc14d24087fa272, 669badac559241e09211e20458ab4122; no voters:
I20260808 10:48:48.914305 30659 raft_consensus.cc:2781] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:48:48.914387 30659 raft_consensus.cc:482] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:48:48.914443 30659 raft_consensus.cc:3037] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:48.915562 30659 raft_consensus.cc:504] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a323a0b51146a4bdc14d24087fa272" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34601 } } peers { permanent_uuid: "669badac559241e09211e20458ab4122" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35151 } } peers { permanent_uuid: "f075a46284e747eea28ef4aa9baf8f07" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 41255 } }
I20260808 10:48:48.915953 30659 leader_election.cc:288] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [CANDIDATE]: Term 1 election: Requested vote from peers 05a323a0b51146a4bdc14d24087fa272 (127.31.237.130:34601), f075a46284e747eea28ef4aa9baf8f07 (127.31.237.129:41255)
I20260808 10:48:48.916348 30346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b1f915c6b04549884c3ef74c8a4bfd" candidate_uuid: "669badac559241e09211e20458ab4122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a323a0b51146a4bdc14d24087fa272"
I20260808 10:48:48.916354 30150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b1f915c6b04549884c3ef74c8a4bfd" candidate_uuid: "669badac559241e09211e20458ab4122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f075a46284e747eea28ef4aa9baf8f07"
I20260808 10:48:48.916484 30346 raft_consensus.cc:3037] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:48.916502 30150 raft_consensus.cc:3037] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:48.917665 30150 raft_consensus.cc:2454] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 669badac559241e09211e20458ab4122 in term 1.
I20260808 10:48:48.917667 30346 raft_consensus.cc:2454] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 669badac559241e09211e20458ab4122 in term 1.
I20260808 10:48:48.917996 30455 leader_election.cc:302] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 669badac559241e09211e20458ab4122, f075a46284e747eea28ef4aa9baf8f07; no voters:
I20260808 10:48:48.918155 30659 raft_consensus.cc:2781] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:48.918371 30659 raft_consensus.cc:686] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [term 1 LEADER]: Becoming Leader. State: Replica: 669badac559241e09211e20458ab4122, State: Running, Role: LEADER
I20260808 10:48:48.918766 30659 consensus_queue.cc:237] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a323a0b51146a4bdc14d24087fa272" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34601 } } peers { permanent_uuid: "669badac559241e09211e20458ab4122" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35151 } } peers { permanent_uuid: "f075a46284e747eea28ef4aa9baf8f07" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 41255 } }
I20260808 10:48:48.922049 29910 catalog_manager.cc:5495] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 reported cstate change: term changed from 0 to 1, leader changed from <none> to 669badac559241e09211e20458ab4122 (127.31.237.131). New cstate: current_term: 1 leader_uuid: "669badac559241e09211e20458ab4122" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a323a0b51146a4bdc14d24087fa272" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "669badac559241e09211e20458ab4122" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 35151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f075a46284e747eea28ef4aa9baf8f07" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 41255 } health_report { overall_health: UNKNOWN } } }
I20260808 10:48:48.938159 30641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.934862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56518 (local address 127.31.237.131:35151)
0808 10:48:48.935011 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:48.935015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:48.935028 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:48:48.935121 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:48.935124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:48.935179 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:48.935263 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:48:48.935491 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.936001 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.936807 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:48.937041 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:48.937124 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:48.937390 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:48:48.937830 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:48:48.937925 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:48.937993 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 10:48:48.938045 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:48:48.944335 30346 raft_consensus.cc:1261] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272 [term 1 FOLLOWER]: Refusing update from remote peer 669badac559241e09211e20458ab4122: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:48:48.944335 30150 raft_consensus.cc:1261] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07 [term 1 FOLLOWER]: Refusing update from remote peer 669badac559241e09211e20458ab4122: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:48:48.945081 30666 consensus_queue.cc:1035] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f075a46284e747eea28ef4aa9baf8f07" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 41255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:48:48.945297 30659 consensus_queue.cc:1035] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a323a0b51146a4bdc14d24087fa272" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:48:48.953742 30673 mvcc.cc:205] Tried to move back new op lower bound from 7316218384148402176 to 7316218384054071296. Current Snapshot: MvccSnapshot[applied={T|T < 7316218384148402176}]
I20260808 10:48:48.954032 30671 mvcc.cc:205] Tried to move back new op lower bound from 7316218384148402176 to 7316218384054071296. Current Snapshot: MvccSnapshot[applied={T|T < 7316218384148402176}]
I20260808 10:48:48.962368 29909 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:48:48.962589 29909 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:48:48.965175 29909 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13d7030cadf945f39798d4a44a2ed171: Sending DeleteTablet for 3 replicas of tablet 26b1f915c6b04549884c3ef74c8a4bfd
I20260808 10:48:48.966080 30316 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b1f915c6b04549884c3ef74c8a4bfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:48 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:43202
I20260808 10:48:48.966117 30122 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b1f915c6b04549884c3ef74c8a4bfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:48 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:41774
I20260808 10:48:48.966110 30525 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b1f915c6b04549884c3ef74c8a4bfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:48 UTC) from {username='kudu', principal='kudu/127.31.237.190@KRBTEST.COM'} at 127.0.0.1:56504
I20260808 10:48:48.966634 30683 tablet_replica.cc:330] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07: stopping tablet replica
I20260808 10:48:48.966813 30684 tablet_replica.cc:330] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272: stopping tablet replica
I20260808 10:48:48.967011 30683 raft_consensus.cc:2229] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:48:48.967026 30682 tablet_replica.cc:330] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122: stopping tablet replica
I20260808 10:48:48.967109 30684 raft_consensus.cc:2229] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:48:48.967291 30682 raft_consensus.cc:2229] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:48:48.967422 30683 raft_consensus.cc:2258] T 26b1f915c6b04549884c3ef74c8a4bfd P f075a46284e747eea28ef4aa9baf8f07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:48.967459 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 30013
I20260808 10:48:48.972782 30684 raft_consensus.cc:2258] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:48.976756 30684 ts_tablet_manager.cc:1905] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:48.979888 30682 raft_consensus.cc:2258] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:48.980039 30684 ts_tablet_manager.cc:1918] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:48.980206 30684 log.cc:1196] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/26b1f915c6b04549884c3ef74c8a4bfd
I20260808 10:48:48.980619 30684 ts_tablet_manager.cc:1939] T 26b1f915c6b04549884c3ef74c8a4bfd P 05a323a0b51146a4bdc14d24087fa272: Deleting consensus metadata
I20260808 10:48:48.981837 30682 ts_tablet_manager.cc:1905] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:48.982148 29883 catalog_manager.cc:4841] TS 05a323a0b51146a4bdc14d24087fa272 (127.31.237.130:34601): tablet 26b1f915c6b04549884c3ef74c8a4bfd (table test-table [id=312049ecdf04433b804913c5259ef36a]) successfully deleted
I20260808 10:48:48.985532 30682 ts_tablet_manager.cc:1918] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:48.985647 30682 log.cc:1196] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/26b1f915c6b04549884c3ef74c8a4bfd
I20260808 10:48:48.986058 30682 ts_tablet_manager.cc:1939] T 26b1f915c6b04549884c3ef74c8a4bfd P 669badac559241e09211e20458ab4122: Deleting consensus metadata
I20260808 10:48:48.987561 29885 catalog_manager.cc:4841] TS 669badac559241e09211e20458ab4122 (127.31.237.131:35151): tablet 26b1f915c6b04549884c3ef74c8a4bfd (table test-table [id=312049ecdf04433b804913c5259ef36a]) successfully deleted
W20260808 10:48:48.988058 29885 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:41255 (error 108)
I20260808 10:48:48.988283 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 30230
I20260808 10:48:48.992962 30638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:48.992495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:41255 (local address 127.0.0.1:41784)
0808 10:48:48.992752 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:48.992855 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:41255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260808 10:48:48.993335 29885 catalog_manager.cc:4584] TS f075a46284e747eea28ef4aa9baf8f07 (127.31.237.129:41255): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26b1f915c6b04549884c3ef74c8a4bfd: Network error: Client connection negotiation failed: client connection to 127.31.237.129:41255: connect: Connection refused (error 111)
I20260808 10:48:49.007735 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 30427
I20260808 10:48:49.031507 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 29856
2026-08-08T10:48:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2615 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T10:48:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:48:49Z Disabled control of system clock
I20260808 10:48:49.073551 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:34315
--webserver_interface=127.31.237.190
--webserver_port=0
--builtin_ntp_servers=127.31.237.148:39267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:34315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:48:49.194742 30692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:49.195027 30692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:49.195096 30692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:49.199357 30692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:48:49.199429 30692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:49.199445 30692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:49.199462 30692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:48:49.199478 30692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:48:49.204535 30692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:39267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:34315
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:34315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:49.205729 30692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:49.207188 30692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:49.216495 30706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:49.216523 30702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:49.216495 30703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:49.216609 30692 server_base.cc:1034] running on GCE node
I20260808 10:48:49.217383 30692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:49.218572 30692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:49.219799 30692 hybrid_clock.cc:648] HybridClock initialized: now 1786186129219770 us; error 51 us; skew 500 ppm
I20260808 10:48:49.225893 30692 webserver.cc:460] Webserver started at http://127.31.237.190:45867/ using document root <none> and password file <none>
I20260808 10:48:49.226658 30692 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:49.226785 30692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:49.227074 30692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:49.229436 30692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "31be3f31f938454c9e8d89892412ba71"
format_stamp: "Formatted at 2026-08-08 10:48:49 on dist-test-slave-twwt"
server_key: "2bd78763a199b2ef4c4d8c549322a488"
server_key_iv: "005680b3ca77cfa510c2f94c66560eca"
server_key_version: "encryptionkey@0"
I20260808 10:48:49.230085 30692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "31be3f31f938454c9e8d89892412ba71"
format_stamp: "Formatted at 2026-08-08 10:48:49 on dist-test-slave-twwt"
server_key: "2bd78763a199b2ef4c4d8c549322a488"
server_key_iv: "005680b3ca77cfa510c2f94c66560eca"
server_key_version: "encryptionkey@0"
I20260808 10:48:49.234251 30692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 10:48:49.237062 30714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:49.238454 30692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:48:49.238667 30692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "31be3f31f938454c9e8d89892412ba71"
format_stamp: "Formatted at 2026-08-08 10:48:49 on dist-test-slave-twwt"
server_key: "2bd78763a199b2ef4c4d8c549322a488"
server_key_iv: "005680b3ca77cfa510c2f94c66560eca"
server_key_version: "encryptionkey@0"
I20260808 10:48:49.238824 30692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:49.256237 30692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:49.257055 30692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:49.257314 30692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:49.265508 30692 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:34315
I20260808 10:48:49.265524 30803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:34315 every 8 connection(s)
I20260808 10:48:49.266572 30692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:48:49.269809 30804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:49.270782 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 30692
I20260808 10:48:49.270923 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:48:49.271273 32694 external_mini_cluster.cc:1428] Setting key 01fdad498bb398c56667a67eb9088ea2
I20260808 10:48:49.276453 30804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71: Bootstrap starting.
I20260808 10:48:49.278815 30804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:49.279572 30804 log.cc:824] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:49.281694 30808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:49.272885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45014 (local address 127.31.237.190:34315)
0808 10:48:49.273621 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:49.273637 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:49.273675 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:48:49.274377 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:49.274397 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:49.274901 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:49.275268 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:49.275759 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.276856 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.277835 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.278354 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.278560 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:49.279551 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:49.279556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:49.279570 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:49.280105 (+ 535us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:49.280191 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:49.280223 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:49.280343 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:49.280574 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 10:48:49.280931 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":147,"threads_started":1}
I20260808 10:48:49.281714 30804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71: No bootstrap required, opened a new log
I20260808 10:48:49.285831 30804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31be3f31f938454c9e8d89892412ba71" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34315 } }
I20260808 10:48:49.286128 30804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:49.286209 30804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31be3f31f938454c9e8d89892412ba71, State: Initialized, Role: FOLLOWER
I20260808 10:48:49.286731 30804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31be3f31f938454c9e8d89892412ba71" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34315 } }
I20260808 10:48:49.286893 30804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:48:49.286962 30804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:48:49.287057 30804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:49.287963 30804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31be3f31f938454c9e8d89892412ba71" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34315 } }
I20260808 10:48:49.288309 30804 leader_election.cc:302] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31be3f31f938454c9e8d89892412ba71; no voters:
I20260808 10:48:49.288655 30804 leader_election.cc:288] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:48:49.288765 30810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:49.289034 30810 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [term 1 LEADER]: Becoming Leader. State: Replica: 31be3f31f938454c9e8d89892412ba71, State: Running, Role: LEADER
I20260808 10:48:49.289433 30810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31be3f31f938454c9e8d89892412ba71" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34315 } }
I20260808 10:48:49.289952 30804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:48:49.291481 30814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31be3f31f938454c9e8d89892412ba71. Latest consensus state: current_term: 1 leader_uuid: "31be3f31f938454c9e8d89892412ba71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31be3f31f938454c9e8d89892412ba71" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34315 } } }
I20260808 10:48:49.291481 30813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31be3f31f938454c9e8d89892412ba71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31be3f31f938454c9e8d89892412ba71" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34315 } } }
I20260808 10:48:49.291648 30814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:49.291648 30813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71 [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:49.292091 30836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:48:49.295964 30836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:48:49.302039 30836 catalog_manager.cc:1347] Generated new cluster ID: 1dade77f07c744d1b7d046c42c90d700
I20260808 10:48:49.302122 30836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:48:49.313247 30836 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:48:49.314301 30836 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:48:49.327653 30836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71: Generated new TSK 0
I20260808 10:48:49.328600 30836 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:48:49.333017 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.129:0
--local_ip_for_outbound_sockets=127.31.237.129
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:34315
--builtin_ntp_servers=127.31.237.148:39267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:48:49.465909 30845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:49.466259 30845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:49.466349 30845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:49.471036 30845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:49.471122 30845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:49.471220 30845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.129
I20260808 10:48:49.476622 30845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:39267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.237.129
--webserver_port=0
--tserver_master_addrs=127.31.237.190:34315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.129
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:49.477854 30845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:49.479444 30845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:49.490381 30857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:49.490442 30845 server_base.cc:1034] running on GCE node
W20260808 10:48:49.490327 30858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:49.490597 30860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:49.491086 30845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:49.491828 30845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:49.493006 30845 hybrid_clock.cc:648] HybridClock initialized: now 1786186129492971 us; error 53 us; skew 500 ppm
I20260808 10:48:49.495137 30845 webserver.cc:460] Webserver started at http://127.31.237.129:40497/ using document root <none> and password file <none>
I20260808 10:48:49.495738 30845 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:49.495803 30845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:49.496112 30845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:49.498332 30845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/data/instance:
uuid: "bec5a401450849fa877131a905bcb635"
format_stamp: "Formatted at 2026-08-08 10:48:49 on dist-test-slave-twwt"
server_key: "b4dcaaa9c66ce7b8f4789c0eee21040d"
server_key_iv: "62d761fd744ec79cb8d85bc5e7b40e33"
server_key_version: "encryptionkey@0"
I20260808 10:48:49.498971 30845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance:
uuid: "bec5a401450849fa877131a905bcb635"
format_stamp: "Formatted at 2026-08-08 10:48:49 on dist-test-slave-twwt"
server_key: "b4dcaaa9c66ce7b8f4789c0eee21040d"
server_key_iv: "62d761fd744ec79cb8d85bc5e7b40e33"
server_key_version: "encryptionkey@0"
I20260808 10:48:49.503345 30845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:48:49.505919 30868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:49.507274 30845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:48:49.507416 30845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/wal
uuid: "bec5a401450849fa877131a905bcb635"
format_stamp: "Formatted at 2026-08-08 10:48:49 on dist-test-slave-twwt"
server_key: "b4dcaaa9c66ce7b8f4789c0eee21040d"
server_key_iv: "62d761fd744ec79cb8d85bc5e7b40e33"
server_key_version: "encryptionkey@0"
I20260808 10:48:49.507530 30845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:49.523316 30845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:49.524101 30845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:49.524314 30845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:49.524946 30845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:49.525972 30845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:49.526044 30845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:49.526118 30845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:49.526168 30845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:49.535581 30845 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.129:45955
I20260808 10:48:49.535634 31040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.129:45955 every 8 connection(s)
I20260808 10:48:49.536530 30845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/data/info.pb
I20260808 10:48:49.541860 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 30845
I20260808 10:48:49.542009 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-0/wal/instance
I20260808 10:48:49.542361 32694 external_mini_cluster.cc:1428] Setting key 9ef68083ec46cd92de52b624c40b2e27
I20260808 10:48:49.544656 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.130:0
--local_ip_for_outbound_sockets=127.31.237.130
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:34315
--builtin_ntp_servers=127.31.237.148:39267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:48:49.546856 30808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:49.538432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.129:40819 (local address 127.31.237.190:34315)
0808 10:48:49.538571 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:49.538575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:49.539650 (+ 1075us) server_negotiation.cc:407] Connection header received
0808 10:48:49.541026 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:49.541029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:49.541076 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:49.541152 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:49.542211 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.542947 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.544508 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.544771 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.544899 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:49.546211 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:49.546216 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:49.546221 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:49.546241 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:49.546272 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:49.546286 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:49.546377 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:49.546596 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 10:48:49.546691 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:48:49.547767 31044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:49.538834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34315 (local address 127.31.237.129:40819)
0808 10:48:49.539497 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:49.539538 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:49.540831 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:49.541284 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:49.541291 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:49.541372 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:49.542049 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:49.542074 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.543080 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.543083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:49.544372 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:49.544385 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.544953 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.544959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:49.545134 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:49.545841 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:48:49.545846 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:49.545911 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:48:49.545998 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:49.546381 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:49.546395 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:49.546404 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:48:49.546587 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 10:48:49.546966 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":118,"threads_started":1}
I20260808 10:48:49.549068 31042 heartbeater.cc:347] Connected to a master server at 127.31.237.190:34315
I20260808 10:48:49.549331 31042 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:49.550024 31042 heartbeater.cc:511] Master 127.31.237.190:34315 requested a full tablet report, sending...
I20260808 10:48:49.551978 30740 ts_manager.cc:194] Registered new tserver with Master: bec5a401450849fa877131a905bcb635 (127.31.237.129:45955)
I20260808 10:48:49.553793 30740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.237.129:40819
W20260808 10:48:49.667145 31046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:49.667424 31046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:49.667464 31046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:49.671557 31046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:49.671622 31046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:49.671676 31046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.130
I20260808 10:48:49.676538 31046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:39267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.237.130
--webserver_port=0
--tserver_master_addrs=127.31.237.190:34315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.130
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:49.677670 31046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:49.679148 31046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:49.690272 31056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:49.690272 31054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:49.690467 31046 server_base.cc:1034] running on GCE node
W20260808 10:48:49.690287 31053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:49.691089 31046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:49.691926 31046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:49.693121 31046 hybrid_clock.cc:648] HybridClock initialized: now 1786186129693084 us; error 50 us; skew 500 ppm
I20260808 10:48:49.695361 31046 webserver.cc:460] Webserver started at http://127.31.237.130:38559/ using document root <none> and password file <none>
I20260808 10:48:49.696058 31046 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:49.696166 31046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:49.696473 31046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:49.699131 31046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/data/instance:
uuid: "22f85acb01c84835af3b3b158d122233"
format_stamp: "Formatted at 2026-08-08 10:48:49 on dist-test-slave-twwt"
server_key: "80e2557d739fdb29d872119b14d431d6"
server_key_iv: "759bdd781b91b8fe975c58928c806301"
server_key_version: "encryptionkey@0"
I20260808 10:48:49.699800 31046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance:
uuid: "22f85acb01c84835af3b3b158d122233"
format_stamp: "Formatted at 2026-08-08 10:48:49 on dist-test-slave-twwt"
server_key: "80e2557d739fdb29d872119b14d431d6"
server_key_iv: "759bdd781b91b8fe975c58928c806301"
server_key_version: "encryptionkey@0"
I20260808 10:48:49.704113 31046 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:48:49.707093 31064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:49.708532 31046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:48:49.708709 31046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/wal
uuid: "22f85acb01c84835af3b3b158d122233"
format_stamp: "Formatted at 2026-08-08 10:48:49 on dist-test-slave-twwt"
server_key: "80e2557d739fdb29d872119b14d431d6"
server_key_iv: "759bdd781b91b8fe975c58928c806301"
server_key_version: "encryptionkey@0"
I20260808 10:48:49.708856 31046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:49.742554 31046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:49.743676 31046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:49.743918 31046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:49.744629 31046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:49.745775 31046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:49.745859 31046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:49.745946 31046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:49.746001 31046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:49.755858 31046 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.130:38175
I20260808 10:48:49.755883 31236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.130:38175 every 8 connection(s)
I20260808 10:48:49.756919 31046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/data/info.pb
I20260808 10:48:49.764099 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 31046
I20260808 10:48:49.764211 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/wal/instance
I20260808 10:48:49.764513 32694 external_mini_cluster.cc:1428] Setting key aac87f5759b5f103f2583bb13efe1bfc
I20260808 10:48:49.766969 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.237.131:0
--local_ip_for_outbound_sockets=127.31.237.131
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:34315
--builtin_ntp_servers=127.31.237.148:39267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:48:49.767325 30808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:49.759021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.130:51419 (local address 127.31.237.190:34315)
0808 10:48:49.759198 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:49.759203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:49.760415 (+ 1212us) server_negotiation.cc:407] Connection header received
0808 10:48:49.761909 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:49.761913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:49.761968 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:49.762066 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:49.763086 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.763619 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.765015 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.765327 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.765436 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:49.766713 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:49.766716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:49.766718 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:49.766731 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:49.766750 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:49.766758 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:49.766818 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:49.767087 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 10:48:49.767175 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 10:48:49.768499 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:49.759446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34315 (local address 127.31.237.130:51419)
0808 10:48:49.760235 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:49.760290 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:49.761678 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:49.762248 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:49.762255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:49.762345 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:49.762948 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:49.762962 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.763739 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.763742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:49.764809 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:49.764820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.765483 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.765487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:49.765641 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:49.766379 (+ 738us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:48:49.766381 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:49.766437 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:48:49.766522 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:49.766851 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:49.766900 (+ 49us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:49.766907 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:48:49.767056 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 10:48:49.767382 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":160,"threads_started":1}
I20260808 10:48:49.769906 31237 heartbeater.cc:347] Connected to a master server at 127.31.237.190:34315
I20260808 10:48:49.770195 31237 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:49.770830 31237 heartbeater.cc:511] Master 127.31.237.190:34315 requested a full tablet report, sending...
I20260808 10:48:49.772035 30740 ts_manager.cc:194] Registered new tserver with Master: 22f85acb01c84835af3b3b158d122233 (127.31.237.130:38175)
I20260808 10:48:49.772949 30740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.237.130:51419
W20260808 10:48:49.885358 31241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:49.885605 31241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:49.885645 31241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:49.889824 31241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:49.889881 31241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:49.889973 31241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.237.131
I20260808 10:48:49.895099 31241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:39267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.237.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.237.131
--webserver_port=0
--tserver_master_addrs=127.31.237.190:34315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.237.131
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:49.896166 31241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:49.897532 31241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:49.907372 31251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:49.907387 31253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:49.907426 31250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:49.907480 31241 server_base.cc:1034] running on GCE node
I20260808 10:48:49.908195 31241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:49.908793 31241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:49.909965 31241 hybrid_clock.cc:648] HybridClock initialized: now 1786186129909949 us; error 26 us; skew 500 ppm
I20260808 10:48:49.911981 31241 webserver.cc:460] Webserver started at http://127.31.237.131:44343/ using document root <none> and password file <none>
I20260808 10:48:49.912626 31241 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:49.912703 31241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:49.912917 31241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:49.915125 31241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/data/instance:
uuid: "6f5fb00cd90148ca8935045479cd7c38"
format_stamp: "Formatted at 2026-08-08 10:48:49 on dist-test-slave-twwt"
server_key: "042293eb23a70ddb7af0ad23970217a0"
server_key_iv: "512c8e42ae0052d6ba524119ca9144a7"
server_key_version: "encryptionkey@0"
I20260808 10:48:49.915696 31241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance:
uuid: "6f5fb00cd90148ca8935045479cd7c38"
format_stamp: "Formatted at 2026-08-08 10:48:49 on dist-test-slave-twwt"
server_key: "042293eb23a70ddb7af0ad23970217a0"
server_key_iv: "512c8e42ae0052d6ba524119ca9144a7"
server_key_version: "encryptionkey@0"
I20260808 10:48:49.919339 31241 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 10:48:49.921746 31262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:49.923048 31241 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:48:49.923188 31241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/wal
uuid: "6f5fb00cd90148ca8935045479cd7c38"
format_stamp: "Formatted at 2026-08-08 10:48:49 on dist-test-slave-twwt"
server_key: "042293eb23a70ddb7af0ad23970217a0"
server_key_iv: "512c8e42ae0052d6ba524119ca9144a7"
server_key_version: "encryptionkey@0"
I20260808 10:48:49.923290 31241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:49.938985 31241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:49.939713 31241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:49.939918 31241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:49.940543 31241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:48:49.941550 31241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:48:49.941597 31241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:49.941669 31241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:48:49.941708 31241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:49.951038 31241 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.131:33205
I20260808 10:48:49.951102 31445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.131:33205 every 8 connection(s)
I20260808 10:48:49.952418 31241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/data/info.pb
I20260808 10:48:49.955516 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 31241
I20260808 10:48:49.956650 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/wal/instance
I20260808 10:48:49.957044 32694 external_mini_cluster.cc:1428] Setting key 2e08b9c1098d27f150da8709bd283d8a
I20260808 10:48:49.963174 30808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:49.956487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:50655 (local address 127.31.237.190:34315)
0808 10:48:49.956612 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:49.956616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:49.956634 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:48:49.958178 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:49.958182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:49.958239 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:49.958313 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:49.959406 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.959952 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.961127 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.961365 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.961480 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:49.962667 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:49.962671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:49.962674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:49.962688 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:49.962717 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:49.962725 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:49.962806 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:49.962971 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 10:48:49.963035 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:48:49.964028 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:49.955279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:34315 (local address 127.31.237.131:50655)
0808 10:48:49.956242 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:49.956303 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:49.957968 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:49.958507 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:49.958516 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:49.958663 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:48:49.959254 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:49.959277 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.960088 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.960091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:49.960995 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:49.961005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.961525 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.961530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:49.961709 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:49.962370 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:48:49.962372 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:48:49.962414 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:48:49.962486 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:48:49.962810 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:48:49.962822 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:48:49.962828 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:48:49.962968 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 10:48:49.963254 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":154,"threads_started":1}
I20260808 10:48:49.965122 31446 heartbeater.cc:347] Connected to a master server at 127.31.237.190:34315
I20260808 10:48:49.965340 31446 heartbeater.cc:464] Registering TS with master...
I20260808 10:48:49.965826 31446 heartbeater.cc:511] Master 127.31.237.190:34315 requested a full tablet report, sending...
I20260808 10:48:49.966847 30740 ts_manager.cc:194] Registered new tserver with Master: 6f5fb00cd90148ca8935045479cd7c38 (127.31.237.131:33205)
I20260808 10:48:49.967451 30740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.237.131:50655
I20260808 10:48:49.971072 32694 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:48:49.976806 30808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:49.973097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45024 (local address 127.31.237.190:34315)
0808 10:48:49.973245 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:49.973250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:49.973471 (+ 221us) server_negotiation.cc:407] Connection header received
0808 10:48:49.973554 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:49.973557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:49.973604 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:49.973700 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:49.973940 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.974438 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.975336 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.975555 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.975638 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:49.976391 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:49.976394 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:49.976396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:49.976409 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:49.976427 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:49.976435 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:49.976493 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:49.976566 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 10:48:49.976618 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:48:49.980100 30740 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:48:49.983309 30740 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:48:49.998380 31463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:49.992228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41920 (local address 127.31.237.129:45955)
0808 10:48:49.993117 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:49.993131 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:49.993156 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:48:49.993239 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:49.993245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:49.993478 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:49.993652 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:49.993994 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.995241 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.996854 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.997639 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.997767 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:49.997956 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:49.998155 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 10:48:49.998238 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":780,"thread_start_us":178,"threads_started":1}
I20260808 10:48:49.998380 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:49.992083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:45955 (local address 127.0.0.1:41920)
0808 10:48:49.992847 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:49.992860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:49.992946 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:49.993662 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:49.993668 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:49.993693 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:49.993859 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:49.993867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.995363 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.995366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:49.996724 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:49.996735 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.997776 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.997782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:49.997910 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:49.997996 (+ 86us) client_negotiation.cc:769] Sending connection context
0808 10:48:49.998116 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 10:48:49.998243 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":476,"thread_start_us":107,"threads_started":1}
I20260808 10:48:49.998875 31465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:49.992965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.131:33205 (local address 127.0.0.1:60280)
0808 10:48:49.993180 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:49.993195 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:49.993376 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:49.994576 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:49.994579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:49.994598 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:49.994817 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:49.994823 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.996089 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.996092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:49.997074 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:49.997082 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.997867 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.997870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:49.997952 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:49.998574 (+ 622us) client_negotiation.cc:769] Sending connection context
0808 10:48:49.998675 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 10:48:49.998742 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":576,"thread_start_us":87,"threads_started":1}
I20260808 10:48:49.999163 31469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:49.993836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60280 (local address 127.31.237.131:33205)
0808 10:48:49.994208 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:49.994219 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:49.994249 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:48:49.994310 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:49.994314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:49.994446 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:49.994600 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:49.994939 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.995946 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.997192 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.997742 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.998024 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:49.998212 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:49.998874 (+ 662us) server_negotiation.cc:299] Negotiation successful
0808 10:48:49.998983 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":189,"threads_started":1}
I20260808 10:48:49.999651 31462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:49.992083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:38175 (local address 127.0.0.1:56978)
0808 10:48:49.992472 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:49.992524 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:49.992631 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:49.994116 (+ 1485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:49.994119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:49.994139 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:49.994231 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:49.994237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.995758 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.995761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:49.996676 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:49.996686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.999276 (+ 2590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.999280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:49.999374 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:49.999401 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 10:48:49.999474 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:48:49.999539 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":93,"threads_started":1}
I20260808 10:48:50.000152 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:49.993284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56978 (local address 127.31.237.130:38175)
0808 10:48:49.993653 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:49.993665 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:49.993688 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:48:49.993762 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:49.993767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:49.993953 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:49.994392 (+ 439us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:49.994525 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.995634 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.996838 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:49.997639 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:49.997767 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:49.997956 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:49.999820 (+ 1864us) server_negotiation.cc:299] Negotiation successful
0808 10:48:49.999927 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":155,"threads_started":1}
I20260808 10:48:50.002142 30939 tablet_service.cc:1467] Processing CreateTablet for tablet a95de17a589748908c3f02fdfa23be4c (DEFAULT_TABLE table=test-table [id=80cc8a5ac7de4ed7a81fa925aa09d404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:50.002187 31335 tablet_service.cc:1467] Processing CreateTablet for tablet a95de17a589748908c3f02fdfa23be4c (DEFAULT_TABLE table=test-table [id=80cc8a5ac7de4ed7a81fa925aa09d404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:50.002681 31138 tablet_service.cc:1467] Processing CreateTablet for tablet a95de17a589748908c3f02fdfa23be4c (DEFAULT_TABLE table=test-table [id=80cc8a5ac7de4ed7a81fa925aa09d404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:48:50.003263 30939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a95de17a589748908c3f02fdfa23be4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:50.003286 31335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a95de17a589748908c3f02fdfa23be4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:50.003608 31138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a95de17a589748908c3f02fdfa23be4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:50.009640 31471 tablet_bootstrap.cc:492] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635: Bootstrap starting.
I20260808 10:48:50.011919 31472 tablet_bootstrap.cc:492] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233: Bootstrap starting.
I20260808 10:48:50.012296 31473 tablet_bootstrap.cc:492] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38: Bootstrap starting.
I20260808 10:48:50.012352 31471 tablet_bootstrap.cc:654] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:50.013465 31471 log.cc:824] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:50.014307 31472 tablet_bootstrap.cc:654] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:50.014671 31473 tablet_bootstrap.cc:654] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:50.015187 31472 log.cc:824] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:50.015475 31473 log.cc:824] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:50.015964 31471 tablet_bootstrap.cc:492] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635: No bootstrap required, opened a new log
I20260808 10:48:50.016189 31471 ts_tablet_manager.cc:1397] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260808 10:48:50.018092 31472 tablet_bootstrap.cc:492] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233: No bootstrap required, opened a new log
I20260808 10:48:50.018298 31472 ts_tablet_manager.cc:1397] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 10:48:50.019987 31473 tablet_bootstrap.cc:492] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38: No bootstrap required, opened a new log
I20260808 10:48:50.020208 31473 ts_tablet_manager.cc:1397] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 10:48:50.020913 31471 raft_consensus.cc:348] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec5a401450849fa877131a905bcb635" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45955 } } peers { permanent_uuid: "22f85acb01c84835af3b3b158d122233" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38175 } } peers { permanent_uuid: "6f5fb00cd90148ca8935045479cd7c38" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 33205 } }
I20260808 10:48:50.021210 31471 raft_consensus.cc:374] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:50.021277 31471 raft_consensus.cc:729] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bec5a401450849fa877131a905bcb635, State: Initialized, Role: FOLLOWER
I20260808 10:48:50.021780 31471 consensus_queue.cc:260] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec5a401450849fa877131a905bcb635" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45955 } } peers { permanent_uuid: "22f85acb01c84835af3b3b158d122233" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38175 } } peers { permanent_uuid: "6f5fb00cd90148ca8935045479cd7c38" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 33205 } }
I20260808 10:48:50.022258 31472 raft_consensus.cc:348] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec5a401450849fa877131a905bcb635" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45955 } } peers { permanent_uuid: "22f85acb01c84835af3b3b158d122233" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38175 } } peers { permanent_uuid: "6f5fb00cd90148ca8935045479cd7c38" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 33205 } }
I20260808 10:48:50.022519 31472 raft_consensus.cc:374] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:50.022576 31472 raft_consensus.cc:729] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f85acb01c84835af3b3b158d122233, State: Initialized, Role: FOLLOWER
I20260808 10:48:50.022579 31042 heartbeater.cc:503] Master 127.31.237.190:34315 was elected leader, sending a full tablet report...
I20260808 10:48:50.022964 31471 ts_tablet_manager.cc:1428] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260808 10:48:50.023084 31472 consensus_queue.cc:260] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec5a401450849fa877131a905bcb635" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45955 } } peers { permanent_uuid: "22f85acb01c84835af3b3b158d122233" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38175 } } peers { permanent_uuid: "6f5fb00cd90148ca8935045479cd7c38" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 33205 } }
I20260808 10:48:50.024065 31472 ts_tablet_manager.cc:1428] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 10:48:50.024413 31237 heartbeater.cc:503] Master 127.31.237.190:34315 was elected leader, sending a full tablet report...
I20260808 10:48:50.024287 31473 raft_consensus.cc:348] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec5a401450849fa877131a905bcb635" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45955 } } peers { permanent_uuid: "22f85acb01c84835af3b3b158d122233" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38175 } } peers { permanent_uuid: "6f5fb00cd90148ca8935045479cd7c38" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 33205 } }
I20260808 10:48:50.024559 31473 raft_consensus.cc:374] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:50.024627 31473 raft_consensus.cc:729] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f5fb00cd90148ca8935045479cd7c38, State: Initialized, Role: FOLLOWER
I20260808 10:48:50.025049 31473 consensus_queue.cc:260] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec5a401450849fa877131a905bcb635" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45955 } } peers { permanent_uuid: "22f85acb01c84835af3b3b158d122233" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38175 } } peers { permanent_uuid: "6f5fb00cd90148ca8935045479cd7c38" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 33205 } }
I20260808 10:48:50.025705 31446 heartbeater.cc:503] Master 127.31.237.190:34315 was elected leader, sending a full tablet report...
I20260808 10:48:50.026042 31473 ts_tablet_manager.cc:1428] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 10:48:50.037918 31043 tablet.cc:2369] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:48:50.116786 31479 raft_consensus.cc:482] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:48:50.117071 31479 raft_consensus.cc:504] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec5a401450849fa877131a905bcb635" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45955 } } peers { permanent_uuid: "22f85acb01c84835af3b3b158d122233" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38175 } } peers { permanent_uuid: "6f5fb00cd90148ca8935045479cd7c38" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 33205 } }
I20260808 10:48:50.118296 31479 leader_election.cc:288] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bec5a401450849fa877131a905bcb635 (127.31.237.129:45955), 22f85acb01c84835af3b3b158d122233 (127.31.237.130:38175)
I20260808 10:48:50.122388 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:50.118567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.130:38175 (local address 127.31.237.131:44077)
0808 10:48:50.118710 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:50.118728 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:50.118837 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:50.119159 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:50.119163 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:50.119180 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:50.119308 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:50.119314 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:50.120300 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:50.120303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:50.121207 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:50.121217 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:50.121928 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:50.121932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:50.122041 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:50.122073 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 10:48:50.122146 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:48:50.122238 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":2}
I20260808 10:48:50.122406 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:50.118823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:44077 (local address 127.31.237.130:38175)
0808 10:48:50.118935 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:50.118938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:50.118950 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:48:50.118999 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:50.119001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:50.119041 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:50.119144 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:50.119492 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:50.120184 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:50.121348 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:50.121798 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:50.121898 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:50.121937 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:50.122180 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 10:48:50.122258 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:48:50.122555 31480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:50.118567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:45955 (local address 127.31.237.131:48925)
0808 10:48:50.119264 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:50.119275 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:48:50.119387 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:48:50.119637 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:48:50.119640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:48:50.119658 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:48:50.119788 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:50.119794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:50.120596 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:50.120598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:50.121416 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:48:50.121426 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:48:50.122112 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:48:50.122115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:48:50.122231 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:50.122252 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:48:50.122339 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:48:50.122399 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":349,"thread_start_us":143,"threads_started":1}
I20260808 10:48:50.122586 31463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:50.118703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.237.131:48925 (local address 127.31.237.129:45955)
0808 10:48:50.118821 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:50.118825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:50.119397 (+ 572us) server_negotiation.cc:407] Connection header received
0808 10:48:50.119496 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:50.119498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:50.119544 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:50.119604 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:48:50.119919 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:50.120485 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:50.121544 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:50.121983 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:50.122101 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:50.122162 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:50.122369 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 10:48:50.122459 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 10:48:50.123159 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95de17a589748908c3f02fdfa23be4c" candidate_uuid: "6f5fb00cd90148ca8935045479cd7c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f85acb01c84835af3b3b158d122233" is_pre_election: true
I20260808 10:48:50.123243 30969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95de17a589748908c3f02fdfa23be4c" candidate_uuid: "6f5fb00cd90148ca8935045479cd7c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bec5a401450849fa877131a905bcb635" is_pre_election: true
I20260808 10:48:50.123512 31165 raft_consensus.cc:2454] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f5fb00cd90148ca8935045479cd7c38 in term 0.
I20260808 10:48:50.123564 30969 raft_consensus.cc:2454] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f5fb00cd90148ca8935045479cd7c38 in term 0.
I20260808 10:48:50.124085 31265 leader_election.cc:302] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f85acb01c84835af3b3b158d122233, 6f5fb00cd90148ca8935045479cd7c38; no voters:
I20260808 10:48:50.124364 31479 raft_consensus.cc:2781] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:48:50.124450 31479 raft_consensus.cc:482] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:48:50.124502 31479 raft_consensus.cc:3037] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:50.125532 31479 raft_consensus.cc:504] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec5a401450849fa877131a905bcb635" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45955 } } peers { permanent_uuid: "22f85acb01c84835af3b3b158d122233" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38175 } } peers { permanent_uuid: "6f5fb00cd90148ca8935045479cd7c38" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 33205 } }
I20260808 10:48:50.125873 31479 leader_election.cc:288] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [CANDIDATE]: Term 1 election: Requested vote from peers bec5a401450849fa877131a905bcb635 (127.31.237.129:45955), 22f85acb01c84835af3b3b158d122233 (127.31.237.130:38175)
I20260808 10:48:50.126215 30969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95de17a589748908c3f02fdfa23be4c" candidate_uuid: "6f5fb00cd90148ca8935045479cd7c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bec5a401450849fa877131a905bcb635"
I20260808 10:48:50.126279 31165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95de17a589748908c3f02fdfa23be4c" candidate_uuid: "6f5fb00cd90148ca8935045479cd7c38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f85acb01c84835af3b3b158d122233"
I20260808 10:48:50.126329 30969 raft_consensus.cc:3037] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:50.126439 31165 raft_consensus.cc:3037] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:50.127573 30969 raft_consensus.cc:2454] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f5fb00cd90148ca8935045479cd7c38 in term 1.
I20260808 10:48:50.127671 31165 raft_consensus.cc:2454] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f5fb00cd90148ca8935045479cd7c38 in term 1.
I20260808 10:48:50.127938 31269 leader_election.cc:302] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f5fb00cd90148ca8935045479cd7c38, bec5a401450849fa877131a905bcb635; no voters:
I20260808 10:48:50.128132 31479 raft_consensus.cc:2781] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:50.128403 31479 raft_consensus.cc:686] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [term 1 LEADER]: Becoming Leader. State: Replica: 6f5fb00cd90148ca8935045479cd7c38, State: Running, Role: LEADER
I20260808 10:48:50.128744 31479 consensus_queue.cc:237] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec5a401450849fa877131a905bcb635" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45955 } } peers { permanent_uuid: "22f85acb01c84835af3b3b158d122233" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38175 } } peers { permanent_uuid: "6f5fb00cd90148ca8935045479cd7c38" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 33205 } }
I20260808 10:48:50.132045 30739 catalog_manager.cc:5495] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f5fb00cd90148ca8935045479cd7c38 (127.31.237.131). New cstate: current_term: 1 leader_uuid: "6f5fb00cd90148ca8935045479cd7c38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec5a401450849fa877131a905bcb635" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f85acb01c84835af3b3b158d122233" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f5fb00cd90148ca8935045479cd7c38" member_type: VOTER last_known_addr { host: "127.31.237.131" port: 33205 } health_report { overall_health: HEALTHY } } }
I20260808 10:48:50.154269 31469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:50.150880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60296 (local address 127.31.237.131:33205)
0808 10:48:50.151032 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:50.151036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:50.151049 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:48:50.151135 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:50.151137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:50.151194 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:50.151286 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:48:50.151525 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:50.152026 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:50.152913 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:50.153130 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:50.153211 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:50.153488 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:48:50.153914 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:48:50.154011 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:50.154087 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 10:48:50.154137 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:48:50.160275 30969 raft_consensus.cc:1261] T a95de17a589748908c3f02fdfa23be4c P bec5a401450849fa877131a905bcb635 [term 1 FOLLOWER]: Refusing update from remote peer 6f5fb00cd90148ca8935045479cd7c38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:48:50.160274 31165 raft_consensus.cc:1261] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233 [term 1 FOLLOWER]: Refusing update from remote peer 6f5fb00cd90148ca8935045479cd7c38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:48:50.161007 31481 consensus_queue.cc:1035] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f85acb01c84835af3b3b158d122233" member_type: VOTER last_known_addr { host: "127.31.237.130" port: 38175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:48:50.161271 31479 consensus_queue.cc:1035] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bec5a401450849fa877131a905bcb635" member_type: VOTER last_known_addr { host: "127.31.237.129" port: 45955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:48:50.169292 31484 mvcc.cc:205] Tried to move back new op lower bound from 7316218389129027584 to 7316218389010198528. Current Snapshot: MvccSnapshot[applied={T|T < 7316218389129027584}]
I20260808 10:48:50.169564 31486 mvcc.cc:205] Tried to move back new op lower bound from 7316218389129027584 to 7316218389010198528. Current Snapshot: MvccSnapshot[applied={T|T < 7316218389129027584}]
I20260808 10:48:50.179154 30739 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:48:50.179351 30739 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:48:50.182014 30739 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 31be3f31f938454c9e8d89892412ba71: Sending DeleteTablet for 3 replicas of tablet a95de17a589748908c3f02fdfa23be4c
I20260808 10:48:50.182982 31335 tablet_service.cc:1514] Processing DeleteTablet for tablet a95de17a589748908c3f02fdfa23be4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:50 UTC) from {username='slave'} at 127.0.0.1:60280
I20260808 10:48:50.183115 31138 tablet_service.cc:1514] Processing DeleteTablet for tablet a95de17a589748908c3f02fdfa23be4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:50 UTC) from {username='slave'} at 127.0.0.1:56978
I20260808 10:48:50.183179 30939 tablet_service.cc:1514] Processing DeleteTablet for tablet a95de17a589748908c3f02fdfa23be4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:48:50 UTC) from {username='slave'} at 127.0.0.1:41920
I20260808 10:48:50.183710 31495 tablet_replica.cc:330] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233: stopping tablet replica
I20260808 10:48:50.183734 31494 tablet_replica.cc:330] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38: stopping tablet replica
I20260808 10:48:50.184001 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 30845
I20260808 10:48:50.184094 31494 raft_consensus.cc:2229] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:48:50.184266 31495 raft_consensus.cc:2229] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:48:50.184500 31494 raft_consensus.cc:2258] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:50.184872 31495 raft_consensus.cc:2258] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:48:50.191721 31494 ts_tablet_manager.cc:1905] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:50.196056 31495 ts_tablet_manager.cc:1905] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:48:50.200807 31494 ts_tablet_manager.cc:1918] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:50.201041 31494 log.cc:1196] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-2/wal/wals/a95de17a589748908c3f02fdfa23be4c
I20260808 10:48:50.201483 31495 ts_tablet_manager.cc:1918] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:48:50.201596 31495 log.cc:1196] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233: Deleting WAL directory at /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786185862616645-32694-0/minicluster-data/ts-1/wal/wals/a95de17a589748908c3f02fdfa23be4c
I20260808 10:48:50.201516 31494 ts_tablet_manager.cc:1939] T a95de17a589748908c3f02fdfa23be4c P 6f5fb00cd90148ca8935045479cd7c38: Deleting consensus metadata
I20260808 10:48:50.201905 31495 ts_tablet_manager.cc:1939] T a95de17a589748908c3f02fdfa23be4c P 22f85acb01c84835af3b3b158d122233: Deleting consensus metadata
I20260808 10:48:50.203243 30717 catalog_manager.cc:4841] TS 22f85acb01c84835af3b3b158d122233 (127.31.237.130:38175): tablet a95de17a589748908c3f02fdfa23be4c (table test-table [id=80cc8a5ac7de4ed7a81fa925aa09d404]) successfully deleted
I20260808 10:48:50.203258 30720 catalog_manager.cc:4841] TS 6f5fb00cd90148ca8935045479cd7c38 (127.31.237.131:33205): tablet a95de17a589748908c3f02fdfa23be4c (table test-table [id=80cc8a5ac7de4ed7a81fa925aa09d404]) successfully deleted
W20260808 10:48:50.209035 30720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.237.129:45955 (error 108)
I20260808 10:48:50.209178 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 31046
I20260808 10:48:50.216900 31462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:50.216506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.129:45955 (local address 127.0.0.1:41932)
0808 10:48:50.216687 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 10:48:50.216777 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.129:45955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260808 10:48:50.217231 30720 catalog_manager.cc:4584] TS bec5a401450849fa877131a905bcb635 (127.31.237.129:45955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a95de17a589748908c3f02fdfa23be4c: Network error: Client connection negotiation failed: client connection to 127.31.237.129:45955: connect: Connection refused (error 111)
I20260808 10:48:50.222395 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 31241
I20260808 10:48:50.235551 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 30692
2026-08-08T10:48:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1199 ms)
[----------] 6 tests from EncryptionPolicyTest (11724 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:48:50 dist-test-slave-twwt krb5kdc[31500](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:48:50 dist-test-slave-twwt krb5kdc[31500](info): set up 2 sockets
Aug 08 10:48:50 dist-test-slave-twwt krb5kdc[31500](info): commencing operation
krb5kdc: starting...
W20260808 10:48:51.485250 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:48:51 dist-test-slave-twwt krb5kdc[31500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:48:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:48:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:48:51.634183 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41237
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:51.751648 31522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:51.752004 31522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:51.752099 31522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:51.756486 31522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:48:51.756611 31522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:51.756649 31522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:51.756692 31522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:48:51.756727 31522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:48:51.762377 31522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41237
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:51.763581 31522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:51.764679 31522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:51.773757 31537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:51.773870 31532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:51.773733 31533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:51.774073 31522 server_base.cc:1034] running on GCE node
I20260808 10:48:51.774783 31522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:51.775949 31522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:51.777146 31522 hybrid_clock.cc:648] HybridClock initialized: now 1786186131777133 us; error 29 us; skew 500 ppm
Aug 08 10:48:51 dist-test-slave-twwt krb5kdc[31500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186131, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:51.780289 31522 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:48:51.781193 31522 webserver.cc:460] Webserver started at http://127.0.0.1:40299/ using document root <none> and password file <none>
I20260808 10:48:51.781728 31522 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:51.781785 31522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:51.781963 31522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:51.784140 31522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "60c7d6857e894b49bbecf5c5503c1e92"
format_stamp: "Formatted at 2026-08-08 10:48:51 on dist-test-slave-twwt"
server_key: "03feb109efc17717ed3b76ba51cd390c"
server_key_iv: "5c9447bd0b639bd06f811d163eee5572"
server_key_version: "encryptionkey@0"
I20260808 10:48:51.784739 31522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "60c7d6857e894b49bbecf5c5503c1e92"
format_stamp: "Formatted at 2026-08-08 10:48:51 on dist-test-slave-twwt"
server_key: "03feb109efc17717ed3b76ba51cd390c"
server_key_iv: "5c9447bd0b639bd06f811d163eee5572"
server_key_version: "encryptionkey@0"
I20260808 10:48:51.788574 31522 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:48:51.791100 31546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:51.792340 31522 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:48:51.792500 31522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "60c7d6857e894b49bbecf5c5503c1e92"
format_stamp: "Formatted at 2026-08-08 10:48:51 on dist-test-slave-twwt"
server_key: "03feb109efc17717ed3b76ba51cd390c"
server_key_iv: "5c9447bd0b639bd06f811d163eee5572"
server_key_version: "encryptionkey@0"
I20260808 10:48:51.792615 31522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:51.805119 31522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:51.808493 31522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:51.808710 31522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:51.816798 31522 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41237
I20260808 10:48:51.816828 31625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41237 every 8 connection(s)
I20260808 10:48:51.817821 31522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:48:51.820799 31626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:51.822835 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 31522
I20260808 10:48:51.823004 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:48:51.823304 32694 external_mini_cluster.cc:1428] Setting key 29d49b23c5eb5d3dc7115c907be71326
I20260808 10:48:51.827661 31626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92: Bootstrap starting.
I20260808 10:48:51.829936 31626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:51.830737 31626 log.cc:824] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:48:51 dist-test-slave-twwt krb5kdc[31500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:48:51.832775 31626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92: No bootstrap required, opened a new log
I20260808 10:48:51.836519 31626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c7d6857e894b49bbecf5c5503c1e92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41237 } }
I20260808 10:48:51.836797 31626 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:51.836867 31626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c7d6857e894b49bbecf5c5503c1e92, State: Initialized, Role: FOLLOWER
I20260808 10:48:51.837458 31626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c7d6857e894b49bbecf5c5503c1e92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41237 } }
I20260808 10:48:51.837634 31626 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:48:51.837700 31626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:48:51.837812 31626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:51.838024 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:51.824651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39721 (local address 127.0.0.1:41237)
0808 10:48:51.825050 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:51.825065 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:51.825102 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:48:51.825750 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:51.825768 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:51.826117 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:51.826460 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:51.827349 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:51.828381 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:51.829247 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:51.829721 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:51.829908 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:51.832784 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:51.832789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:51.832803 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:51.832831 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:51.834551 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:51.835095 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:51.835108 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:51.835113 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:51.835245 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:51.835612 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:51.835621 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:51.835623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:51.836015 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:51.836275 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:51.836607 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 10:48:51.836884 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":115,"threads_started":1}
I20260808 10:48:51.838913 31626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c7d6857e894b49bbecf5c5503c1e92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41237 } }
I20260808 10:48:51.839286 31626 leader_election.cc:302] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60c7d6857e894b49bbecf5c5503c1e92; no voters:
I20260808 10:48:51.839644 31626 leader_election.cc:288] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:48:51.839766 31634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:51.840008 31634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [term 1 LEADER]: Becoming Leader. State: Replica: 60c7d6857e894b49bbecf5c5503c1e92, State: Running, Role: LEADER
I20260808 10:48:51.840317 31634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c7d6857e894b49bbecf5c5503c1e92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41237 } }
I20260808 10:48:51.840979 31626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:48:51.841423 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:48:51.841558 31634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60c7d6857e894b49bbecf5c5503c1e92. Latest consensus state: current_term: 1 leader_uuid: "60c7d6857e894b49bbecf5c5503c1e92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c7d6857e894b49bbecf5c5503c1e92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41237 } } }
I20260808 10:48:51.841665 31634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:51.842058 31643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:48:51.842267 31637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60c7d6857e894b49bbecf5c5503c1e92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c7d6857e894b49bbecf5c5503c1e92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41237 } } }
I20260808 10:48:51.842435 31637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92 [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:51.845484 31643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:48:51.851248 31643 catalog_manager.cc:1347] Generated new cluster ID: fdeafda975d4428289863e6917773005
I20260808 10:48:51.851312 31643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:48:51.853045 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:51.843707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55481 (local address 127.0.0.1:41237)
0808 10:48:51.843812 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:51.843816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:51.844086 (+ 270us) server_negotiation.cc:407] Connection header received
0808 10:48:51.844234 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:51.844237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:51.844283 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:51.844362 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:51.845194 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:51.845740 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:51.846980 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:51.847304 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:51.847408 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:51.849306 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:51.849310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:51.849313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:51.849339 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:51.851065 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:51.851639 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:51.851643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:51.851644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:51.851738 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:51.852145 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:51.852148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:51.852150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:51.852332 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:51.852480 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:51.852803 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 10:48:51.852899 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:48:51.864094 31643 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:48:51.865316 31643 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:48:51.881793 31643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60c7d6857e894b49bbecf5c5503c1e92: Generated new TSK 0
I20260808 10:48:51.882663 31643 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:48:51.891690 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 31522
W20260808 10:48:51.904633 31498 connection.cc:535] client connection to 127.0.0.1:41237 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41237): Connection reset by peer (error 104)
2026-08-08T10:48:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1656 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:48:51 dist-test-slave-twwt krb5kdc[31679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:48:51 dist-test-slave-twwt krb5kdc[31679](info): set up 2 sockets
Aug 08 10:48:51 dist-test-slave-twwt krb5kdc[31679](info): commencing operation
krb5kdc: starting...
W20260808 10:48:53.202244 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:48:53 dist-test-slave-twwt krb5kdc[31679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:48:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:48:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:48:53.339037 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33359
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:53.456101 31700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:53.456365 31700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:53.456449 31700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:53.460577 31700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:48:53.460638 31700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:53.460657 31700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:53.460675 31700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:48:53.460727 31700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:48:53.466054 31700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33359
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:53.467170 31700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:53.468282 31700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:48:53.477046 31700 server_base.cc:1034] running on GCE node
W20260808 10:48:53.476974 31708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:53.476974 31711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:53.477051 31709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:53.477833 31700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:53.478951 31700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:53.480399 31700 hybrid_clock.cc:648] HybridClock initialized: now 1786186133480364 us; error 44 us; skew 500 ppm
Aug 08 10:48:53 dist-test-slave-twwt krb5kdc[31679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186133, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:53.483256 31700 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:48:53.484148 31700 webserver.cc:460] Webserver started at http://127.0.0.1:46319/ using document root <none> and password file <none>
I20260808 10:48:53.484733 31700 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:53.484824 31700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:53.485060 31700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:53.487178 31700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "fa28879346074b2c8cc70e986914cee2"
format_stamp: "Formatted at 2026-08-08 10:48:53 on dist-test-slave-twwt"
server_key: "57d75a104a31925a38d1316a93cc49b2"
server_key_iv: "fca6f2187ee1b0d79d510e9706f72fbe"
server_key_version: "encryptionkey@0"
I20260808 10:48:53.487754 31700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "fa28879346074b2c8cc70e986914cee2"
format_stamp: "Formatted at 2026-08-08 10:48:53 on dist-test-slave-twwt"
server_key: "57d75a104a31925a38d1316a93cc49b2"
server_key_iv: "fca6f2187ee1b0d79d510e9706f72fbe"
server_key_version: "encryptionkey@0"
I20260808 10:48:53.491568 31700 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:48:53.494144 31722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:53.495493 31700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 10:48:53.495645 31700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "fa28879346074b2c8cc70e986914cee2"
format_stamp: "Formatted at 2026-08-08 10:48:53 on dist-test-slave-twwt"
server_key: "57d75a104a31925a38d1316a93cc49b2"
server_key_iv: "fca6f2187ee1b0d79d510e9706f72fbe"
server_key_version: "encryptionkey@0"
I20260808 10:48:53.495764 31700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:53.515398 31700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:53.518781 31700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:53.518998 31700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:53.526572 31700 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33359
I20260808 10:48:53.526599 31802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33359 every 8 connection(s)
I20260808 10:48:53.527580 31700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:48:53.530493 31804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:53.535965 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 31700
I20260808 10:48:53.536121 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:48:53.536368 31804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2: Bootstrap starting.
I20260808 10:48:53.536465 32694 external_mini_cluster.cc:1428] Setting key 7dfd703a601bb87012fb1b40b9e66398
I20260808 10:48:53.539007 31804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:53.539738 31804 log.cc:824] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:53.541589 31804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2: No bootstrap required, opened a new log
I20260808 10:48:53.545320 31804 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa28879346074b2c8cc70e986914cee2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33359 } }
I20260808 10:48:53.545481 31807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:53.537893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51925 (local address 127.0.0.1:33359)
0808 10:48:53.538341 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:53.538357 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:53.538393 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:48:53.539138 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:53.539159 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:53.539506 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:53.539842 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:53.540266 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:53.541220 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:53.542186 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:53.542733 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:53.542904 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:53.543830 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:53.543835 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:53.543847 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:53.544221 (+ 374us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:53.544275 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:53.544299 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:53.544447 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:53.544628 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 10:48:53.544765 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":130,"threads_started":1}
I20260808 10:48:53.545569 31804 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:53.545666 31804 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa28879346074b2c8cc70e986914cee2, State: Initialized, Role: FOLLOWER
I20260808 10:48:53.546134 31804 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa28879346074b2c8cc70e986914cee2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33359 } }
I20260808 10:48:53.546278 31804 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:48:53.546343 31804 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:48:53.546427 31804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:53.547367 31804 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa28879346074b2c8cc70e986914cee2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33359 } }
I20260808 10:48:53.547719 31804 leader_election.cc:302] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa28879346074b2c8cc70e986914cee2; no voters:
I20260808 10:48:53.548072 31804 leader_election.cc:288] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:48:53.548199 31809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:53.548414 31809 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [term 1 LEADER]: Becoming Leader. State: Replica: fa28879346074b2c8cc70e986914cee2, State: Running, Role: LEADER
I20260808 10:48:53.548712 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:48:53.548830 31809 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa28879346074b2c8cc70e986914cee2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33359 } }
I20260808 10:48:53.549448 31804 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:48:53.550328 31811 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa28879346074b2c8cc70e986914cee2. Latest consensus state: current_term: 1 leader_uuid: "fa28879346074b2c8cc70e986914cee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa28879346074b2c8cc70e986914cee2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33359 } } }
I20260808 10:48:53.550462 31811 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:53.550808 31810 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa28879346074b2c8cc70e986914cee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa28879346074b2c8cc70e986914cee2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33359 } } }
I20260808 10:48:53.550897 31810 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2 [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:53.551219 31829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:48:53.554878 31829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:48:53.556612 31807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:53.551286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45269 (local address 127.0.0.1:33359)
0808 10:48:53.551410 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:53.551417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:53.552105 (+ 688us) server_negotiation.cc:407] Connection header received
0808 10:48:53.552267 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:53.552270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:53.552325 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:53.552377 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:53.552770 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:53.553499 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:53.554670 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:53.554998 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:53.555111 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:53.556068 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:53.556072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:53.556075 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:53.556088 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:53.556119 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:53.556132 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:53.556232 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:53.556430 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 10:48:53.556474 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 10:48:53.560571 31829 catalog_manager.cc:1347] Generated new cluster ID: 56b6e8ced14d4032a96c010050a3e6eb
I20260808 10:48:53.560642 31829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:48:53.587414 31829 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:48:53.588514 31829 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:48:53.593174 31829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa28879346074b2c8cc70e986914cee2: Generated new TSK 0
I20260808 10:48:53.593915 31829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:48:53.605840 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 31700
W20260808 10:48:53.623904 31677 connection.cc:535] client connection to 127.0.0.1:33359 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33359): Connection reset by peer (error 104)
2026-08-08T10:48:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1719 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:48:53 dist-test-slave-twwt krb5kdc[31850](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:48:53 dist-test-slave-twwt krb5kdc[31850](info): set up 2 sockets
Aug 08 10:48:53 dist-test-slave-twwt krb5kdc[31850](info): commencing operation
krb5kdc: starting...
W20260808 10:48:54.893537 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:48:54 dist-test-slave-twwt krb5kdc[31850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:48:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:48:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:48:55.033601 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42509
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:55.152539 31880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:55.152944 31880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:55.153059 31880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:55.157177 31880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:48:55.157274 31880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:55.157313 31880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:55.157369 31880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:48:55.157409 31880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:48:55.162935 31880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42509
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:55.164050 31880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:55.165123 31880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:55.174588 31889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:55.174599 31887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:55.174669 31880 server_base.cc:1034] running on GCE node
W20260808 10:48:55.174638 31886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:55.175501 31880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:55.176782 31880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:55.178004 31880 hybrid_clock.cc:648] HybridClock initialized: now 1786186135177986 us; error 30 us; skew 500 ppm
Aug 08 10:48:55 dist-test-slave-twwt krb5kdc[31850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186135, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:55.186067 31880 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:48:55.187218 31880 webserver.cc:460] Webserver started at http://127.0.0.1:36053/ using document root <none> and password file <none>
I20260808 10:48:55.187844 31880 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:55.187942 31880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:55.188190 31880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:55.190320 31880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "d4aa6284041c4575a33c7966307f863e"
format_stamp: "Formatted at 2026-08-08 10:48:55 on dist-test-slave-twwt"
server_key: "a8afefc21d50ae7cd5fc59d9de867d19"
server_key_iv: "8b66e040eba38efb9d71aca3bcd97e73"
server_key_version: "encryptionkey@0"
I20260808 10:48:55.190958 31880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "d4aa6284041c4575a33c7966307f863e"
format_stamp: "Formatted at 2026-08-08 10:48:55 on dist-test-slave-twwt"
server_key: "a8afefc21d50ae7cd5fc59d9de867d19"
server_key_iv: "8b66e040eba38efb9d71aca3bcd97e73"
server_key_version: "encryptionkey@0"
I20260808 10:48:55.194964 31880 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260808 10:48:55.197538 31896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:55.198930 31880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:48:55.199090 31880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "d4aa6284041c4575a33c7966307f863e"
format_stamp: "Formatted at 2026-08-08 10:48:55 on dist-test-slave-twwt"
server_key: "a8afefc21d50ae7cd5fc59d9de867d19"
server_key_iv: "8b66e040eba38efb9d71aca3bcd97e73"
server_key_version: "encryptionkey@0"
I20260808 10:48:55.199208 31880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:55.203949 31880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:55.207166 31880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:55.207366 31880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:55.215057 31880 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42509
I20260808 10:48:55.215085 31983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42509 every 8 connection(s)
I20260808 10:48:55.216274 31880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:48:55.219388 31984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:55.221397 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 31880
I20260808 10:48:55.221645 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:48:55.222015 32694 external_mini_cluster.cc:1428] Setting key 8285c5e8377a8456ffd673f3f4ac5733
I20260808 10:48:55.226176 31984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e: Bootstrap starting.
I20260808 10:48:55.228122 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:55.223573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35875 (local address 127.0.0.1:42509)
0808 10:48:55.224054 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:55.224077 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:55.224126 (+ 49us) server_negotiation.cc:407] Connection header received
0808 10:48:55.224982 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:55.225008 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:55.225401 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:55.225782 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:55.226579 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:55.226585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:55.226600 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:55.226638 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:55.226706 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:55.226735 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:55.226881 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:55.226995 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 10:48:55.227149 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":134,"threads_started":1}
I20260808 10:48:55.229111 31984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:55.229959 31984 log.cc:824] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:55.231283 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:48:55.232281 31984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e: No bootstrap required, opened a new log
I20260808 10:48:55.235320 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:55.233435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47661 (local address 127.0.0.1:42509)
0808 10:48:55.233552 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:55.233556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:55.233734 (+ 178us) server_negotiation.cc:407] Connection header received
0808 10:48:55.233896 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:55.233899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:55.233938 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:55.234018 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:55.234884 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:55.234888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:55.234891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:55.234905 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:55.234936 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:55.234946 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:55.235057 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:55.235133 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 10:48:55.235177 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:48:55.236795 31984 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4aa6284041c4575a33c7966307f863e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260808 10:48:55.237032 31984 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:55.237120 31984 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4aa6284041c4575a33c7966307f863e, State: Initialized, Role: FOLLOWER
I20260808 10:48:55.237622 31984 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4aa6284041c4575a33c7966307f863e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260808 10:48:55.237777 31984 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:48:55.237849 31984 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:48:55.237972 31984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:55.239094 31984 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4aa6284041c4575a33c7966307f863e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260808 10:48:55.239509 31984 leader_election.cc:302] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4aa6284041c4575a33c7966307f863e; no voters:
I20260808 10:48:55.239850 31984 leader_election.cc:288] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:48:55.239996 31996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:55.240228 31996 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [term 1 LEADER]: Becoming Leader. State: Replica: d4aa6284041c4575a33c7966307f863e, State: Running, Role: LEADER
I20260808 10:48:55.240581 31996 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4aa6284041c4575a33c7966307f863e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260808 10:48:55.241169 31984 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:48:55.242341 32000 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4aa6284041c4575a33c7966307f863e. Latest consensus state: current_term: 1 leader_uuid: "d4aa6284041c4575a33c7966307f863e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4aa6284041c4575a33c7966307f863e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } } }
I20260808 10:48:55.242479 32000 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:55.242785 31999 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4aa6284041c4575a33c7966307f863e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4aa6284041c4575a33c7966307f863e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } } }
I20260808 10:48:55.242934 31999 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:55.243244 32012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:48:55.246023 32012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:48:55.251608 32012 catalog_manager.cc:1347] Generated new cluster ID: 8bbeff9019e8472ea36732c10bf152f5
I20260808 10:48:55.251688 32012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:48:55.274756 32012 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:48:55.275125 32012 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:48:55.279450 32012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4aa6284041c4575a33c7966307f863e: Generated new TSK 0
I20260808 10:48:55.280237 32012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:48:55.286294 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 31880
2026-08-08T10:48:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1673 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:48:55 dist-test-slave-twwt krb5kdc[32029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:48:55 dist-test-slave-twwt krb5kdc[32029](info): set up 2 sockets
Aug 08 10:48:55 dist-test-slave-twwt krb5kdc[32029](info): commencing operation
krb5kdc: starting...
W20260808 10:48:56.537796 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:48:56 dist-test-slave-twwt krb5kdc[32029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:48:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:48:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:48:56.678678 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46479
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:56.800396 32049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:56.800735 32049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:56.800820 32049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:56.804967 32049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:48:56.805054 32049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:56.805094 32049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:56.805148 32049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:48:56.805186 32049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:48:56.811326 32049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:56.812480 32049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:56.813629 32049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:56.822768 32058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:56.822803 32059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:56.822758 32061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:56.822821 32049 server_base.cc:1034] running on GCE node
I20260808 10:48:56.823642 32049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:56.824714 32049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:56.825949 32049 hybrid_clock.cc:648] HybridClock initialized: now 1786186136825917 us; error 45 us; skew 500 ppm
Aug 08 10:48:56 dist-test-slave-twwt krb5kdc[32029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186136, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:56.828867 32049 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:48:56.829766 32049 webserver.cc:460] Webserver started at http://127.0.0.1:40565/ using document root <none> and password file <none>
I20260808 10:48:56.830363 32049 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:56.830446 32049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:56.830754 32049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:56.832837 32049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "c2607b1b7b2148cdbb83408502783417"
format_stamp: "Formatted at 2026-08-08 10:48:56 on dist-test-slave-twwt"
server_key: "8fccc6069af63cf1395eaf99fd8783e8"
server_key_iv: "e083412b2d4705ad370f53687eec2189"
server_key_version: "encryptionkey@0"
I20260808 10:48:56.833421 32049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "c2607b1b7b2148cdbb83408502783417"
format_stamp: "Formatted at 2026-08-08 10:48:56 on dist-test-slave-twwt"
server_key: "8fccc6069af63cf1395eaf99fd8783e8"
server_key_iv: "e083412b2d4705ad370f53687eec2189"
server_key_version: "encryptionkey@0"
I20260808 10:48:56.837213 32049 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 10:48:56.839663 32073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:56.840950 32049 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:48:56.841084 32049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "c2607b1b7b2148cdbb83408502783417"
format_stamp: "Formatted at 2026-08-08 10:48:56 on dist-test-slave-twwt"
server_key: "8fccc6069af63cf1395eaf99fd8783e8"
server_key_iv: "e083412b2d4705ad370f53687eec2189"
server_key_version: "encryptionkey@0"
I20260808 10:48:56.841190 32049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:56.876638 32049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:56.880185 32049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:56.880369 32049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:56.888008 32049 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46479
I20260808 10:48:56.888036 32156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46479 every 8 connection(s)
I20260808 10:48:56.889102 32049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:48:56.892233 32157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:56.897612 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 32049
I20260808 10:48:56.897765 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:48:56.898065 32694 external_mini_cluster.cc:1428] Setting key a5e6ec2cb0dc16db137485b3d7ada9c2
I20260808 10:48:56.898340 32157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417: Bootstrap starting.
I20260808 10:48:56.901077 32157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:56.901866 32157 log.cc:824] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:56.903817 32157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417: No bootstrap required, opened a new log
Aug 08 10:48:56 dist-test-slave-twwt krb5kdc[32029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:48:56.907716 32157 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2607b1b7b2148cdbb83408502783417" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46479 } }
I20260808 10:48:56.908010 32157 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:56.908095 32157 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2607b1b7b2148cdbb83408502783417, State: Initialized, Role: FOLLOWER
I20260808 10:48:56.908620 32157 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2607b1b7b2148cdbb83408502783417" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46479 } }
I20260808 10:48:56.908772 32157 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:48:56.908839 32157 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:48:56.908969 32157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:56.909888 32157 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2607b1b7b2148cdbb83408502783417" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46479 } }
I20260808 10:48:56.910279 32157 leader_election.cc:302] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2607b1b7b2148cdbb83408502783417; no voters:
I20260808 10:48:56.910696 32157 leader_election.cc:288] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:48:56.910785 32162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:56.911108 32162 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [term 1 LEADER]: Becoming Leader. State: Replica: c2607b1b7b2148cdbb83408502783417, State: Running, Role: LEADER
I20260808 10:48:56.911450 32162 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2607b1b7b2148cdbb83408502783417" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46479 } }
I20260808 10:48:56.912132 32157 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:48:56.912642 32162 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2607b1b7b2148cdbb83408502783417. Latest consensus state: current_term: 1 leader_uuid: "c2607b1b7b2148cdbb83408502783417" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2607b1b7b2148cdbb83408502783417" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46479 } } }
I20260808 10:48:56.912788 32162 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:56.913120 32163 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2607b1b7b2148cdbb83408502783417" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2607b1b7b2148cdbb83408502783417" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46479 } } }
I20260808 10:48:56.913303 32163 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417 [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:56.913815 32160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:56.899536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33225 (local address 127.0.0.1:46479)
0808 10:48:56.900058 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:56.900074 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:56.900114 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:48:56.900852 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:56.900873 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:56.901263 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:56.901633 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:56.902483 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:56.903531 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:56.904474 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:56.904948 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:56.905133 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:56.907978 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:56.908004 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:56.908025 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:56.908070 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:56.910088 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:56.910640 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:56.910649 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:56.910659 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:56.910822 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:56.911188 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:56.911191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:56.911193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:56.911703 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:56.911884 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:56.912413 (+ 529us) server_negotiation.cc:299] Negotiation successful
0808 10:48:56.912765 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":171,"threads_started":1}
I20260808 10:48:56.917568 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 10:48:56.917759 32183 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:48:56.917822 32183 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:48:56.917914 32167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:48:56.918689 32167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:48:56.924971 32167 catalog_manager.cc:1347] Generated new cluster ID: 08d829cb6fdd4a4fbff8af4e035ebb19
I20260808 10:48:56.925045 32167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:48:56.928951 32160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:56.920047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46279 (local address 127.0.0.1:46479)
0808 10:48:56.920767 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:56.920772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:56.920788 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:48:56.920826 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:56.920829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:56.920874 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:56.920960 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:56.921850 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:56.922572 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:56.923636 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:56.924008 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:56.924123 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:56.925698 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:56.925702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:56.925705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:48:56.925737 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:56.927234 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:56.927683 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:56.927687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:56.927689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:56.927773 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:48:56.928053 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:48:56.928057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:48:56.928059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:48:56.928239 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:56.928381 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:56.928624 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 10:48:56.928758 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652}
I20260808 10:48:56.962123 32167 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:48:56.963459 32167 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:48:56.971015 32167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2607b1b7b2148cdbb83408502783417: Generated new TSK 0
I20260808 10:48:56.971778 32167 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:48:56.975562 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 32049
2026-08-08T10:48:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1690 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:48:57 dist-test-slave-twwt krb5kdc[32197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:48:57 dist-test-slave-twwt krb5kdc[32197](info): set up 2 sockets
Aug 08 10:48:57 dist-test-slave-twwt krb5kdc[32197](info): commencing operation
krb5kdc: starting...
W20260808 10:48:58.262864 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:48:58 dist-test-slave-twwt krb5kdc[32197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:48:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:48:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:48:58.406464 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35383
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:48:58.527966 32220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:48:58.528261 32220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:48:58.528306 32220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:48:58.532363 32220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:48:58.532434 32220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:48:58.532454 32220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:48:58.532470 32220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:48:58.532485 32220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:48:58.537880 32220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:48:58.539023 32220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:48:58.540056 32220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:48:58.549119 32236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:58.549113 32228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:48:58.549113 32227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:48:58.549599 32220 server_base.cc:1034] running on GCE node
I20260808 10:48:58.550232 32220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:48:58.551463 32220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:48:58.552692 32220 hybrid_clock.cc:648] HybridClock initialized: now 1786186138552671 us; error 33 us; skew 500 ppm
Aug 08 10:48:58 dist-test-slave-twwt krb5kdc[32197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186138, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:48:58.555830 32220 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:48:58.556849 32220 webserver.cc:460] Webserver started at http://127.0.0.1:35335/ using document root <none> and password file <none>
I20260808 10:48:58.557456 32220 fs_manager.cc:359] Metadata directory not provided
I20260808 10:48:58.557555 32220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:48:58.557818 32220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:48:58.560082 32220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "d7fc5107eaf94fa399c64dc342616f4c"
format_stamp: "Formatted at 2026-08-08 10:48:58 on dist-test-slave-twwt"
server_key: "c2241c712ac0d040f651b5a6f6286662"
server_key_iv: "decdbaed0c7db96a1c475b9d0aa84f19"
server_key_version: "encryptionkey@0"
I20260808 10:48:58.560695 32220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "d7fc5107eaf94fa399c64dc342616f4c"
format_stamp: "Formatted at 2026-08-08 10:48:58 on dist-test-slave-twwt"
server_key: "c2241c712ac0d040f651b5a6f6286662"
server_key_iv: "decdbaed0c7db96a1c475b9d0aa84f19"
server_key_version: "encryptionkey@0"
I20260808 10:48:58.564810 32220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:48:58.567945 32248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:48:58.569356 32220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:48:58.569522 32220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "d7fc5107eaf94fa399c64dc342616f4c"
format_stamp: "Formatted at 2026-08-08 10:48:58 on dist-test-slave-twwt"
server_key: "c2241c712ac0d040f651b5a6f6286662"
server_key_iv: "decdbaed0c7db96a1c475b9d0aa84f19"
server_key_version: "encryptionkey@0"
I20260808 10:48:58.569653 32220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:48:58.589366 32220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:48:58.593298 32220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:48:58.593482 32220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:48:58.602029 32220 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35383
I20260808 10:48:58.602043 32331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35383 every 8 connection(s)
I20260808 10:48:58.603349 32220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:48:58.604931 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 32220
I20260808 10:48:58.605096 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:48:58.605458 32694 external_mini_cluster.cc:1428] Setting key e80e365b00eafa6adc7b9f8cdc024c48
I20260808 10:48:58.606762 32332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:48:58.613144 32332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c: Bootstrap starting.
I20260808 10:48:58.615291 32335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:58.607084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39329 (local address 127.0.0.1:35383)
0808 10:48:58.607568 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:58.607586 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:58.607625 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:48:58.608333 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:58.608352 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:58.608727 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:58.609055 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:58.609459 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:58.610559 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:58.611629 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:58.612168 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:58.612364 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:58.613355 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:58.613361 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:58.613375 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:58.613819 (+ 444us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:58.613882 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:58.613908 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:58.614033 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:58.614179 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 10:48:58.614419 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":171,"threads_started":1}
I20260808 10:48:58.615703 32332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c: Neither blocks nor log segments found. Creating new log.
I20260808 10:48:58.616516 32332 log.cc:824] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:48:58.618803 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:48:58.619024 32332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c: No bootstrap required, opened a new log
I20260808 10:48:58.623416 32332 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fc5107eaf94fa399c64dc342616f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35383 } }
I20260808 10:48:58.623713 32332 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:48:58.623803 32332 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7fc5107eaf94fa399c64dc342616f4c, State: Initialized, Role: FOLLOWER
I20260808 10:48:58.624338 32332 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fc5107eaf94fa399c64dc342616f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35383 } }
I20260808 10:48:58.624544 32332 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:48:58.624626 32332 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:48:58.624735 32332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:48:58.625907 32335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:48:58.621273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54965 (local address 127.0.0.1:35383)
0808 10:48:58.621396 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 10:48:58.621401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:48:58.621594 (+ 193us) server_negotiation.cc:407] Connection header received
0808 10:48:58.621711 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:48:58.621715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:48:58.621766 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:48:58.621863 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:48:58.622186 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:58.622982 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:58.624020 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:48:58.624361 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:48:58.624473 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:48:58.625455 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:48:58.625459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:48:58.625463 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:48:58.625476 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:48:58.625507 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:48:58.625520 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:48:58.625599 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:48:58.625681 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 10:48:58.625744 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:48:58.626132 32332 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fc5107eaf94fa399c64dc342616f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35383 } }
I20260808 10:48:58.626530 32332 leader_election.cc:302] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7fc5107eaf94fa399c64dc342616f4c; no voters:
I20260808 10:48:58.626958 32332 leader_election.cc:288] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:48:58.627094 32347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:48:58.627373 32347 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [term 1 LEADER]: Becoming Leader. State: Replica: d7fc5107eaf94fa399c64dc342616f4c, State: Running, Role: LEADER
I20260808 10:48:58.627812 32347 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fc5107eaf94fa399c64dc342616f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35383 } }
I20260808 10:48:58.628400 32332 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:48:58.629624 32348 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7fc5107eaf94fa399c64dc342616f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fc5107eaf94fa399c64dc342616f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35383 } } }
I20260808 10:48:58.629758 32348 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:58.630189 32358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:48:58.630244 32349 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7fc5107eaf94fa399c64dc342616f4c. Latest consensus state: current_term: 1 leader_uuid: "d7fc5107eaf94fa399c64dc342616f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fc5107eaf94fa399c64dc342616f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35383 } } }
I20260808 10:48:58.630415 32349 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c [sys.catalog]: This master's current role is: LEADER
I20260808 10:48:58.633978 32358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:48:58.640998 32358 catalog_manager.cc:1347] Generated new cluster ID: 5cf9ba1bf2824d779c59fc548bf24308
I20260808 10:48:58.641090 32358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:48:58.660269 32358 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:48:58.661909 32358 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:48:58.668059 32358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7fc5107eaf94fa399c64dc342616f4c: Generated new TSK 0
I20260808 10:48:58.669026 32358 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:48:58.684087 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 32220
2026-08-08T10:48:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1710 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:48:58 dist-test-slave-twwt krb5kdc[32372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:48:58 dist-test-slave-twwt krb5kdc[32372](info): set up 2 sockets
Aug 08 10:48:58 dist-test-slave-twwt krb5kdc[32372](info): commencing operation
krb5kdc: starting...
W20260808 10:48:59.945137 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:00 dist-test-slave-twwt krb5kdc[32372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:49:00.091122 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33139
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:00.208032 32390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:00.208316 32390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:00.208360 32390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:00.212359 32390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:00.212438 32390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:00.212460 32390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:00.212477 32390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:00.212493 32390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:00.218477 32390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:00.219677 32390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:00.221014 32390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:00.230307 32403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:00.230307 32401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:00.230331 32399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:00.230327 32390 server_base.cc:1034] running on GCE node
I20260808 10:49:00.231258 32390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:00.232481 32390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:00.233659 32390 hybrid_clock.cc:648] HybridClock initialized: now 1786186140233625 us; error 45 us; skew 500 ppm
Aug 08 10:49:00 dist-test-slave-twwt krb5kdc[32372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186140, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:00.241622 32390 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:49:00.242835 32390 webserver.cc:460] Webserver started at http://127.0.0.1:37881/ using document root <none> and password file <none>
I20260808 10:49:00.243472 32390 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:00.243571 32390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:00.243836 32390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:00.246042 32390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "04e4de0a1be94695a1efa10405ad9b27"
format_stamp: "Formatted at 2026-08-08 10:49:00 on dist-test-slave-twwt"
server_key: "64a71a2d2ea483faf28efbef0f7ca9bd"
server_key_iv: "df9a273dc61ebcd00f6be0712db74198"
server_key_version: "encryptionkey@0"
I20260808 10:49:00.246702 32390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "04e4de0a1be94695a1efa10405ad9b27"
format_stamp: "Formatted at 2026-08-08 10:49:00 on dist-test-slave-twwt"
server_key: "64a71a2d2ea483faf28efbef0f7ca9bd"
server_key_iv: "df9a273dc61ebcd00f6be0712db74198"
server_key_version: "encryptionkey@0"
I20260808 10:49:00.250694 32390 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:49:00.253259 32416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:00.254491 32390 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 10:49:00.254678 32390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "04e4de0a1be94695a1efa10405ad9b27"
format_stamp: "Formatted at 2026-08-08 10:49:00 on dist-test-slave-twwt"
server_key: "64a71a2d2ea483faf28efbef0f7ca9bd"
server_key_iv: "df9a273dc61ebcd00f6be0712db74198"
server_key_version: "encryptionkey@0"
I20260808 10:49:00.254802 32390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:00.259562 32390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:00.263015 32390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:00.263216 32390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:00.270882 32390 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33139
I20260808 10:49:00.270903 32506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33139 every 8 connection(s)
I20260808 10:49:00.271889 32390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:00.274781 32508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:00.279083 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 32390
I20260808 10:49:00.279233 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:00.279528 32694 external_mini_cluster.cc:1428] Setting key 4e8d3007048ea9d0d8a4d1c525568397
I20260808 10:49:00.280861 32508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27: Bootstrap starting.
I20260808 10:49:00.283260 32508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:00.284085 32508 log.cc:824] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:00.285348 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:00.281060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43395 (local address 127.0.0.1:33139)
0808 10:49:00.281531 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:00.281547 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:00.281594 (+ 47us) server_negotiation.cc:407] Connection header received
0808 10:49:00.282250 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:00.282268 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:00.282574 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:00.282991 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:00.283838 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:00.283844 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:00.283865 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:00.283876 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:00.283930 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:00.283950 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:00.284066 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:00.284173 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 10:49:00.284501 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":143,"threads_started":1}
I20260808 10:49:00.286360 32508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27: No bootstrap required, opened a new log
I20260808 10:49:00.288367 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:00.290491 32508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e4de0a1be94695a1efa10405ad9b27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33139 } }
I20260808 10:49:00.290830 32508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:49:00.290913 32508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04e4de0a1be94695a1efa10405ad9b27, State: Initialized, Role: FOLLOWER
I20260808 10:49:00.291460 32508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e4de0a1be94695a1efa10405ad9b27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33139 } }
I20260808 10:49:00.291646 32508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:49:00.291734 32508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:49:00.291862 32508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:00.292281 32511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:00.290473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36403 (local address 127.0.0.1:33139)
0808 10:49:00.290652 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:00.290658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:00.290871 (+ 213us) server_negotiation.cc:407] Connection header received
0808 10:49:00.290949 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:00.290954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:00.291018 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:00.291118 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:00.291901 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:00.291904 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:00.291907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:00.291922 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:00.291957 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:00.291971 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:00.292064 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:00.292124 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 10:49:00.292162 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:49:00.292981 32508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e4de0a1be94695a1efa10405ad9b27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33139 } }
I20260808 10:49:00.293365 32508 leader_election.cc:302] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04e4de0a1be94695a1efa10405ad9b27; no voters:
I20260808 10:49:00.293702 32508 leader_election.cc:288] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:49:00.293771 32523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:00.294054 32523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [term 1 LEADER]: Becoming Leader. State: Replica: 04e4de0a1be94695a1efa10405ad9b27, State: Running, Role: LEADER
I20260808 10:49:00.294452 32523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e4de0a1be94695a1efa10405ad9b27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33139 } }
I20260808 10:49:00.295090 32508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:00.296242 32525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04e4de0a1be94695a1efa10405ad9b27. Latest consensus state: current_term: 1 leader_uuid: "04e4de0a1be94695a1efa10405ad9b27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e4de0a1be94695a1efa10405ad9b27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33139 } } }
I20260808 10:49:00.296389 32525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:00.296725 32524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04e4de0a1be94695a1efa10405ad9b27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e4de0a1be94695a1efa10405ad9b27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33139 } } }
I20260808 10:49:00.296808 32534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:00.296880 32524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:00.299715 32534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:00.305527 32534 catalog_manager.cc:1347] Generated new cluster ID: 9cabb15029354cec9ee255cbdd0121c6
I20260808 10:49:00.305600 32534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:00.334259 32534 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:00.334728 32534 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:00.347859 32534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04e4de0a1be94695a1efa10405ad9b27: Generated new TSK 0
I20260808 10:49:00.348641 32534 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:49:00.361920 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 32390
2026-08-08T10:49:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1679 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:49:00 dist-test-slave-twwt krb5kdc[32552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:49:00 dist-test-slave-twwt krb5kdc[32552](info): set up 2 sockets
Aug 08 10:49:00 dist-test-slave-twwt krb5kdc[32552](info): commencing operation
krb5kdc: starting...
W20260808 10:49:01.613210 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:01 dist-test-slave-twwt krb5kdc[32552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:49:01.750391 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44829
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:01.868494 32572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:01.868824 32572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:01.868945 32572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:01.873107 32572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:01.873221 32572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:01.873258 32572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:01.873297 32572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:01.873355 32572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:01.879047 32572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:01.880261 32572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:01.881495 32572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:01.890311 32582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:01.890323 32581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:01.890383 32584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:01.890478 32572 server_base.cc:1034] running on GCE node
I20260808 10:49:01.891223 32572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:01.892463 32572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:01.893652 32572 hybrid_clock.cc:648] HybridClock initialized: now 1786186141893638 us; error 29 us; skew 500 ppm
Aug 08 10:49:01 dist-test-slave-twwt krb5kdc[32552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186141, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:01.896628 32572 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:49:01.897609 32572 webserver.cc:460] Webserver started at http://127.0.0.1:38083/ using document root <none> and password file <none>
I20260808 10:49:01.898202 32572 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:01.898296 32572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:01.898542 32572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:01.900705 32572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "e4e7081c757d49d0b88271f0501d526f"
format_stamp: "Formatted at 2026-08-08 10:49:01 on dist-test-slave-twwt"
server_key: "d4a5c5b481c6f0618977315fc3384ecf"
server_key_iv: "afe8fcd8550aabcdc60b8f950abe753b"
server_key_version: "encryptionkey@0"
I20260808 10:49:01.901312 32572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "e4e7081c757d49d0b88271f0501d526f"
format_stamp: "Formatted at 2026-08-08 10:49:01 on dist-test-slave-twwt"
server_key: "d4a5c5b481c6f0618977315fc3384ecf"
server_key_iv: "afe8fcd8550aabcdc60b8f950abe753b"
server_key_version: "encryptionkey@0"
I20260808 10:49:01.905155 32572 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:49:01.907784 32594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:01.909179 32572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:49:01.909310 32572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "e4e7081c757d49d0b88271f0501d526f"
format_stamp: "Formatted at 2026-08-08 10:49:01 on dist-test-slave-twwt"
server_key: "d4a5c5b481c6f0618977315fc3384ecf"
server_key_iv: "afe8fcd8550aabcdc60b8f950abe753b"
server_key_version: "encryptionkey@0"
I20260808 10:49:01.909421 32572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:01.920888 32572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:01.924408 32572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:01.924654 32572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:01.932629 32572 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44829
I20260808 10:49:01.932657 32687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44829 every 8 connection(s)
I20260808 10:49:01.933740 32572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:01.936945 32690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:01.938199 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 32572
I20260808 10:49:01.938383 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:01.938800 32694 external_mini_cluster.cc:1428] Setting key fe8fef9eabecda4ba35d1b75e91264e5
I20260808 10:49:01.943827 32690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f: Bootstrap starting.
I20260808 10:49:01.946779 32690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:01.947832 32690 log.cc:824] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:49:01 dist-test-slave-twwt krb5kdc[32552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:49:01.950443 32690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f: No bootstrap required, opened a new log
I20260808 10:49:01.956285 32690 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e7081c757d49d0b88271f0501d526f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44829 } }
I20260808 10:49:01.956652 32690 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:49:01.956718 32690 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4e7081c757d49d0b88271f0501d526f, State: Initialized, Role: FOLLOWER
I20260808 10:49:01.956785 32693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:01.940523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:42199 (local address 127.0.0.1:44829)
0808 10:49:01.941031 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:01.941060 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:01.941100 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:49:01.941816 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:01.941836 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:01.942222 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:01.942549 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:01.943557 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:01.944728 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:01.945998 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:01.946537 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:01.946767 (+ 230us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:01.949949 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:01.949978 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:01.950001 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:01.950042 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:01.952668 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:01.953295 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:01.953301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:01.953308 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:01.953496 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:01.954026 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:01.954029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:01.954031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:01.954735 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:01.955036 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:01.955390 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 10:49:01.955665 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":176,"threads_started":1}
I20260808 10:49:01.957296 32690 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e7081c757d49d0b88271f0501d526f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44829 } }
I20260808 10:49:01.957522 32690 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:49:01.957603 32690 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:49:01.957716 32690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:01.958928 32690 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e7081c757d49d0b88271f0501d526f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44829 } }
I20260808 10:49:01.959309 32690 leader_election.cc:302] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4e7081c757d49d0b88271f0501d526f; no voters:
I20260808 10:49:01.959733 32690 leader_election.cc:288] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:49:01.959868 32702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:01.960080 32702 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [term 1 LEADER]: Becoming Leader. State: Replica: e4e7081c757d49d0b88271f0501d526f, State: Running, Role: LEADER
I20260808 10:49:01.960475 32702 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e7081c757d49d0b88271f0501d526f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44829 } }
I20260808 10:49:01.960947 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:01.961174 32690 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:01.962225 32704 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4e7081c757d49d0b88271f0501d526f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e7081c757d49d0b88271f0501d526f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44829 } } }
I20260808 10:49:01.962355 32704 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:01.962731 32709 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4e7081c757d49d0b88271f0501d526f. Latest consensus state: current_term: 1 leader_uuid: "e4e7081c757d49d0b88271f0501d526f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4e7081c757d49d0b88271f0501d526f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44829 } } }
I20260808 10:49:01.962842 32709 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:01.963121 32726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:01.966511 32726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:01.972564 32693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:01.964275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:48151 (local address 127.0.0.1:44829)
0808 10:49:01.964390 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:01.964396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:01.964634 (+ 238us) server_negotiation.cc:407] Connection header received
0808 10:49:01.964802 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:01.964805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:01.964856 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:01.964949 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:01.965628 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:01.966166 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:01.967091 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:01.967345 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:01.967460 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:01.968891 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:01.968895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:01.968898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:01.968922 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:01.970434 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:01.970994 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:01.970998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:01.971000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:01.971098 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:01.971509 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:01.971512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:01.971514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:01.971729 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:01.971853 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:01.972304 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 10:49:01.972413 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:49:01.973057 32726 catalog_manager.cc:1347] Generated new cluster ID: 7308855c2c6247338ed6129a5d96cecc
I20260808 10:49:01.973126 32726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:01.992453 32726 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:01.993816 32726 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:01.998660 32726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4e7081c757d49d0b88271f0501d526f: Generated new TSK 0
I20260808 10:49:01.999388 32726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:49:02.007668 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 32572
W20260808 10:49:02.021739 32549 connection.cc:535] client connection to 127.0.0.1:44829 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44829): Connection reset by peer (error 104)
2026-08-08T10:49:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1643 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:49:02 dist-test-slave-twwt krb5kdc[32743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:49:02 dist-test-slave-twwt krb5kdc[32743](info): set up 2 sockets
Aug 08 10:49:02 dist-test-slave-twwt krb5kdc[32743](info): commencing operation
krb5kdc: starting...
W20260808 10:49:03.285352 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:03 dist-test-slave-twwt krb5kdc[32743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:49:03.422991 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42749
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:03.541450 32764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:03.541791 32764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:03.541872 32764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:03.545986 32764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:03.546092 32764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:03.546133 32764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:03.546190 32764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:03.546231 32764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:03.551733 32764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:03.552945 32764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:03.554034 32764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:03.563539 308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:03.563566 305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:03.563570 310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:03.564033 32764 server_base.cc:1034] running on GCE node
I20260808 10:49:03.564672 32764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:03.565893 32764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:03.567098 32764 hybrid_clock.cc:648] HybridClock initialized: now 1786186143567074 us; error 35 us; skew 500 ppm
Aug 08 10:49:03 dist-test-slave-twwt krb5kdc[32743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186143, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:03.569974 32764 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:49:03.571098 32764 webserver.cc:460] Webserver started at http://127.0.0.1:36273/ using document root <none> and password file <none>
I20260808 10:49:03.571709 32764 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:03.571806 32764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:03.572067 32764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:03.574312 32764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "24424944de9748e8bb0f54d9cbe79212"
format_stamp: "Formatted at 2026-08-08 10:49:03 on dist-test-slave-twwt"
server_key: "d85d39616eb7012472f948ff75e53338"
server_key_iv: "3ffc0584d3d832df98483feec398bced"
server_key_version: "encryptionkey@0"
I20260808 10:49:03.574971 32764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "24424944de9748e8bb0f54d9cbe79212"
format_stamp: "Formatted at 2026-08-08 10:49:03 on dist-test-slave-twwt"
server_key: "d85d39616eb7012472f948ff75e53338"
server_key_iv: "3ffc0584d3d832df98483feec398bced"
server_key_version: "encryptionkey@0"
I20260808 10:49:03.579011 32764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 10:49:03.581585 325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:03.582999 32764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:49:03.583166 32764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "24424944de9748e8bb0f54d9cbe79212"
format_stamp: "Formatted at 2026-08-08 10:49:03 on dist-test-slave-twwt"
server_key: "d85d39616eb7012472f948ff75e53338"
server_key_iv: "3ffc0584d3d832df98483feec398bced"
server_key_version: "encryptionkey@0"
I20260808 10:49:03.583293 32764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:03.599442 32764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:03.602442 32764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:03.602682 32764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:03.609998 32764 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42749
I20260808 10:49:03.610028 406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42749 every 8 connection(s)
I20260808 10:49:03.611028 32764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:03.613883 407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:03.619457 407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212: Bootstrap starting.
I20260808 10:49:03.621205 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 32764
I20260808 10:49:03.621367 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:03.621678 32694 external_mini_cluster.cc:1428] Setting key f277134b449d2b0e58d362d55fcf1912
I20260808 10:49:03.622198 407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:03.623123 407 log.cc:824] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:03.625188 407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212: No bootstrap required, opened a new log
I20260808 10:49:03.629490 407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24424944de9748e8bb0f54d9cbe79212" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42749 } }
I20260808 10:49:03.629756 407 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:49:03.629820 407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24424944de9748e8bb0f54d9cbe79212, State: Initialized, Role: FOLLOWER
I20260808 10:49:03.630326 407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24424944de9748e8bb0f54d9cbe79212" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42749 } }
I20260808 10:49:03.630467 407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:49:03.630577 407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:49:03.630712 407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:03.631073 412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:03.622978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:53911 (local address 127.0.0.1:42749)
0808 10:49:03.623479 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:03.623497 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:03.623534 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:49:03.624198 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:03.624217 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:03.624562 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:03.624847 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:03.625195 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:03.626241 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:03.627511 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:03.627993 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:03.628114 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:03.629090 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:03.629095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:03.629108 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:03.629505 (+ 397us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:03.629563 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:03.629585 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:03.629687 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:03.629983 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 10:49:03.630133 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":161,"threads_started":1}
I20260808 10:49:03.631696 407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24424944de9748e8bb0f54d9cbe79212" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42749 } }
I20260808 10:49:03.632073 407 leader_election.cc:302] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24424944de9748e8bb0f54d9cbe79212; no voters:
I20260808 10:49:03.632429 407 leader_election.cc:288] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:49:03.632534 414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:03.632772 414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [term 1 LEADER]: Becoming Leader. State: Replica: 24424944de9748e8bb0f54d9cbe79212, State: Running, Role: LEADER
I20260808 10:49:03.633102 414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24424944de9748e8bb0f54d9cbe79212" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42749 } }
I20260808 10:49:03.633795 407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:03.634632 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:03.634912 416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24424944de9748e8bb0f54d9cbe79212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24424944de9748e8bb0f54d9cbe79212" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42749 } } }
I20260808 10:49:03.635025 416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:03.635414 430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:03.636219 417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24424944de9748e8bb0f54d9cbe79212. Latest consensus state: current_term: 1 leader_uuid: "24424944de9748e8bb0f54d9cbe79212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24424944de9748e8bb0f54d9cbe79212" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42749 } } }
I20260808 10:49:03.636373 417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:03.639533 430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:03.642000 412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:03.637059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:34443 (local address 127.0.0.1:42749)
0808 10:49:03.637181 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:03.637185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:03.637427 (+ 242us) server_negotiation.cc:407] Connection header received
0808 10:49:03.637640 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:03.637643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:03.637702 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:03.637797 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:03.638103 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:03.638868 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:03.640005 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:03.640358 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:03.640477 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:03.641457 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:03.641462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:03.641465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:03.641483 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:03.641515 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:03.641529 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:03.641626 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:03.641807 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 10:49:03.641848 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:49:03.646351 430 catalog_manager.cc:1347] Generated new cluster ID: 5a3ecdb716d24e97aa38a86334528af7
I20260808 10:49:03.646466 430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:03.669644 430 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:03.670761 430 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:03.680266 430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24424944de9748e8bb0f54d9cbe79212: Generated new TSK 0
I20260808 10:49:03.681015 430 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:49:03.687568 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 32764
W20260808 10:49:03.703860 32739 connection.cc:535] client connection to 127.0.0.1:42749 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42749): Connection reset by peer (error 104)
2026-08-08T10:49:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1681 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:49:03 dist-test-slave-twwt krb5kdc[451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:49:03 dist-test-slave-twwt krb5kdc[451](info): set up 2 sockets
Aug 08 10:49:03 dist-test-slave-twwt krb5kdc[451](info): commencing operation
krb5kdc: starting...
W20260808 10:49:04.966549 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:05 dist-test-slave-twwt krb5kdc[451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:49:05.106132 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45529
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:05.226826 469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:05.227119 469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:05.227161 469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:05.231262 469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:05.231334 469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:05.231354 469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:05.231371 469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:05.231386 469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:05.237074 469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:05.238281 469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:05.239436 469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:05.248803 480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:05.248828 478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:05.248905 475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:05.249001 469 server_base.cc:1034] running on GCE node
I20260808 10:49:05.249764 469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:05.251010 469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:05.252188 469 hybrid_clock.cc:648] HybridClock initialized: now 1786186145252153 us; error 48 us; skew 500 ppm
Aug 08 10:49:05 dist-test-slave-twwt krb5kdc[451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186145, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:05.261324 469 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:49:05.262463 469 webserver.cc:460] Webserver started at http://127.0.0.1:45627/ using document root <none> and password file <none>
I20260808 10:49:05.263175 469 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:05.263262 469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:05.263459 469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:05.265790 469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "b026bc7ba11548fbb3cdef326ee5de94"
format_stamp: "Formatted at 2026-08-08 10:49:05 on dist-test-slave-twwt"
server_key: "ca9e96c0217c418865273ecb9d5ac2ee"
server_key_iv: "a4ee33994514671a511547a2c46a10ce"
server_key_version: "encryptionkey@0"
I20260808 10:49:05.266383 469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "b026bc7ba11548fbb3cdef326ee5de94"
format_stamp: "Formatted at 2026-08-08 10:49:05 on dist-test-slave-twwt"
server_key: "ca9e96c0217c418865273ecb9d5ac2ee"
server_key_iv: "a4ee33994514671a511547a2c46a10ce"
server_key_version: "encryptionkey@0"
I20260808 10:49:05.270691 469 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:49:05.273471 487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:05.275020 469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:49:05.275149 469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "b026bc7ba11548fbb3cdef326ee5de94"
format_stamp: "Formatted at 2026-08-08 10:49:05 on dist-test-slave-twwt"
server_key: "ca9e96c0217c418865273ecb9d5ac2ee"
server_key_iv: "a4ee33994514671a511547a2c46a10ce"
server_key_version: "encryptionkey@0"
I20260808 10:49:05.275247 469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:05.280056 469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:05.283418 469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:05.283577 469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:05.290779 469 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45529
I20260808 10:49:05.290820 572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45529 every 8 connection(s)
I20260808 10:49:05.291769 469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:05.294391 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 469
I20260808 10:49:05.294541 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:05.294946 575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:05.294946 32694 external_mini_cluster.cc:1428] Setting key e0b4bcea0b566ba24f0d14e1b770e8c4
I20260808 10:49:05.300789 581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:05.296488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:45709 (local address 127.0.0.1:45529)
0808 10:49:05.297028 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:05.297057 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:05.297097 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:49:05.297955 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:05.297978 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:05.298293 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:05.298702 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:05.299560 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:05.299566 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:05.299578 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:05.299590 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:05.299647 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:05.299667 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:05.299830 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:05.299914 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 10:49:05.300080 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":155,"threads_started":1}
I20260808 10:49:05.302217 575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94: Bootstrap starting.
I20260808 10:49:05.303959 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:05.304903 575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:05.305727 575 log.cc:824] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:05.307857 581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:05.306170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:45979 (local address 127.0.0.1:45529)
0808 10:49:05.306308 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:05.306313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:05.306462 (+ 149us) server_negotiation.cc:407] Connection header received
0808 10:49:05.306547 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:05.306553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:05.306594 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:05.306708 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:05.307436 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:05.307440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:05.307442 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:05.307454 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:05.307486 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:05.307496 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:05.307586 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:05.307689 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 10:49:05.307733 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:49:05.308368 575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94: No bootstrap required, opened a new log
I20260808 10:49:05.312354 575 raft_consensus.cc:348] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b026bc7ba11548fbb3cdef326ee5de94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45529 } }
I20260808 10:49:05.312609 575 raft_consensus.cc:374] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:49:05.312685 575 raft_consensus.cc:729] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b026bc7ba11548fbb3cdef326ee5de94, State: Initialized, Role: FOLLOWER
I20260808 10:49:05.313201 575 consensus_queue.cc:260] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b026bc7ba11548fbb3cdef326ee5de94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45529 } }
I20260808 10:49:05.313377 575 raft_consensus.cc:388] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:49:05.313468 575 raft_consensus.cc:482] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:49:05.313594 575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:05.314591 575 raft_consensus.cc:504] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b026bc7ba11548fbb3cdef326ee5de94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45529 } }
I20260808 10:49:05.315023 575 leader_election.cc:302] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b026bc7ba11548fbb3cdef326ee5de94; no voters:
I20260808 10:49:05.315415 575 leader_election.cc:288] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:49:05.315562 594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:05.315840 594 raft_consensus.cc:686] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [term 1 LEADER]: Becoming Leader. State: Replica: b026bc7ba11548fbb3cdef326ee5de94, State: Running, Role: LEADER
I20260808 10:49:05.316220 594 consensus_queue.cc:237] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b026bc7ba11548fbb3cdef326ee5de94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45529 } }
I20260808 10:49:05.316799 575 sys_catalog.cc:564] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:05.318289 596 sys_catalog.cc:455] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b026bc7ba11548fbb3cdef326ee5de94. Latest consensus state: current_term: 1 leader_uuid: "b026bc7ba11548fbb3cdef326ee5de94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b026bc7ba11548fbb3cdef326ee5de94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45529 } } }
I20260808 10:49:05.318301 595 sys_catalog.cc:455] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b026bc7ba11548fbb3cdef326ee5de94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b026bc7ba11548fbb3cdef326ee5de94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45529 } } }
I20260808 10:49:05.318465 595 sys_catalog.cc:458] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:05.318784 596 sys_catalog.cc:458] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:05.318987 615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:05.322052 615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:05.328262 615 catalog_manager.cc:1347] Generated new cluster ID: 992173ce79f0402cab2ea191712319c2
I20260808 10:49:05.328380 615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:05.345470 615 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:05.345841 615 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:05.351068 615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b026bc7ba11548fbb3cdef326ee5de94: Generated new TSK 0
I20260808 10:49:05.351969 615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:49:05.364426 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 469
2026-08-08T10:49:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1672 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:49:05 dist-test-slave-twwt krb5kdc[633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:49:05 dist-test-slave-twwt krb5kdc[633](info): set up 2 sockets
Aug 08 10:49:05 dist-test-slave-twwt krb5kdc[633](info): commencing operation
krb5kdc: starting...
W20260808 10:49:06.653205 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:06 dist-test-slave-twwt krb5kdc[633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:49:06.790110 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37419
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:06.907884 658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:06.908175 658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:06.908221 658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:06.912813 658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:06.912882 658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:06.912902 658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:06.912919 658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:06.912935 658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:06.918306 658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37419
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:06.919456 658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:06.920456 658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:06.929245 671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:06.929234 665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:06.929234 667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:06.929508 658 server_base.cc:1034] running on GCE node
I20260808 10:49:06.930199 658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:06.931313 658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:06.932494 658 hybrid_clock.cc:648] HybridClock initialized: now 1786186146932463 us; error 43 us; skew 500 ppm
Aug 08 10:49:06 dist-test-slave-twwt krb5kdc[633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:06.935544 658 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:49:06.936589 658 webserver.cc:460] Webserver started at http://127.0.0.1:33809/ using document root <none> and password file <none>
I20260808 10:49:06.937294 658 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:06.937368 658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:06.937615 658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:06.939960 658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "433385842c0a4c79901b4b3a97eccdaf"
format_stamp: "Formatted at 2026-08-08 10:49:06 on dist-test-slave-twwt"
server_key: "80bd17524e352337eaea43eb033cc6d6"
server_key_iv: "c2b86cc3feeebb6aff52ae732a6c0875"
server_key_version: "encryptionkey@0"
I20260808 10:49:06.940586 658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "433385842c0a4c79901b4b3a97eccdaf"
format_stamp: "Formatted at 2026-08-08 10:49:06 on dist-test-slave-twwt"
server_key: "80bd17524e352337eaea43eb033cc6d6"
server_key_iv: "c2b86cc3feeebb6aff52ae732a6c0875"
server_key_version: "encryptionkey@0"
I20260808 10:49:06.944962 658 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:49:06.947911 681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:06.949328 658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:49:06.949496 658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "433385842c0a4c79901b4b3a97eccdaf"
format_stamp: "Formatted at 2026-08-08 10:49:06 on dist-test-slave-twwt"
server_key: "80bd17524e352337eaea43eb033cc6d6"
server_key_iv: "c2b86cc3feeebb6aff52ae732a6c0875"
server_key_version: "encryptionkey@0"
I20260808 10:49:06.949621 658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:06.963630 658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:06.966768 658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:06.966987 658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:06.974445 658 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37419
I20260808 10:49:06.974506 760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37419 every 8 connection(s)
I20260808 10:49:06.975476 658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:06.978034 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 658
I20260808 10:49:06.978209 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:06.978439 762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:06.978535 32694 external_mini_cluster.cc:1428] Setting key aa973d78641f091dc0c069c12916ecfc
I20260808 10:49:06.985216 762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf: Bootstrap starting.
I20260808 10:49:06.987973 762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf: Neither blocks nor log segments found. Creating new log.
Aug 08 10:49:06 dist-test-slave-twwt krb5kdc[633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:49:06.988763 762 log.cc:824] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:06.991149 762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf: No bootstrap required, opened a new log
I20260808 10:49:06.993947 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:06.979937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:34883 (local address 127.0.0.1:37419)
0808 10:49:06.980441 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:06.980456 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:06.980502 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:49:06.981385 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:06.981407 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:06.981760 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:06.982136 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:06.983057 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:06.984073 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:06.984996 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:06.985532 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:06.985716 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:06.988748 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:06.988753 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:06.988768 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:06.988806 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:06.990811 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:06.991395 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:06.991400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:06.991405 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:06.991516 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:06.991900 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:06.991903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:06.991904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:06.992295 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:06.992537 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:06.992896 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 10:49:06.993132 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":160,"threads_started":1}
I20260808 10:49:06.995280 762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433385842c0a4c79901b4b3a97eccdaf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37419 } }
I20260808 10:49:06.995532 762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:49:06.995612 762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 433385842c0a4c79901b4b3a97eccdaf, State: Initialized, Role: FOLLOWER
I20260808 10:49:06.996120 762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433385842c0a4c79901b4b3a97eccdaf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37419 } }
I20260808 10:49:06.996287 762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:49:06.996376 762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:49:06.996483 762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:06.997274 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:06.997594 762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433385842c0a4c79901b4b3a97eccdaf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37419 } }
I20260808 10:49:06.997948 762 leader_election.cc:302] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 433385842c0a4c79901b4b3a97eccdaf; no voters:
I20260808 10:49:06.998296 762 leader_election.cc:288] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:49:06.998377 769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:06.998572 769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [term 1 LEADER]: Becoming Leader. State: Replica: 433385842c0a4c79901b4b3a97eccdaf, State: Running, Role: LEADER
I20260808 10:49:06.998894 769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433385842c0a4c79901b4b3a97eccdaf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37419 } }
I20260808 10:49:06.999630 762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:07.000794 772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 433385842c0a4c79901b4b3a97eccdaf. Latest consensus state: current_term: 1 leader_uuid: "433385842c0a4c79901b4b3a97eccdaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433385842c0a4c79901b4b3a97eccdaf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37419 } } }
I20260808 10:49:07.000946 772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:07.001287 774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "433385842c0a4c79901b4b3a97eccdaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433385842c0a4c79901b4b3a97eccdaf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37419 } } }
I20260808 10:49:07.001415 787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:07.001447 774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:07.005499 787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:07.012131 787 catalog_manager.cc:1347] Generated new cluster ID: 3abb204032ae40808bd5522b4c2b89bb
I20260808 10:49:07.012272 787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:07.013356 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:07.002307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:54707 (local address 127.0.0.1:37419)
0808 10:49:07.002447 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:07.002453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:07.003002 (+ 549us) server_negotiation.cc:407] Connection header received
0808 10:49:07.003295 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:07.003299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:07.003353 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:07.003411 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:07.004400 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:07.005240 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:07.006429 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:07.006832 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:07.006947 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:07.008747 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:07.008752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:07.008755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:07.008784 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:07.010775 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:07.011359 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:07.011364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:07.011366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:07.011463 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:07.011858 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:07.011862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:07.011863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:07.012096 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:07.012952 (+ 856us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:07.013060 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 10:49:07.013166 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:49:07.035346 787 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:07.036413 787 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:07.041270 787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 433385842c0a4c79901b4b3a97eccdaf: Generated new TSK 0
I20260808 10:49:07.042217 787 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:49:07.053128 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 658
W20260808 10:49:07.069041 630 connection.cc:535] client connection to 127.0.0.1:37419 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37419): Connection reset by peer (error 104)
2026-08-08T10:49:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1693 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:49:07 dist-test-slave-twwt krb5kdc[810](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:49:07 dist-test-slave-twwt krb5kdc[810](info): set up 2 sockets
Aug 08 10:49:07 dist-test-slave-twwt krb5kdc[810](info): commencing operation
krb5kdc: starting...
W20260808 10:49:08.426342 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.342s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:08 dist-test-slave-twwt krb5kdc[810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:49:08.562456 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:08.684523 828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:08.684831 828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:08.684875 828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:08.689030 828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:08.689100 828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:08.689121 828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:08.689137 828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:08.689152 828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:08.694793 828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34323
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:08.695993 828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:08.697099 828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:08.706516 839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:08.706589 836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:08.706516 837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:08.706887 828 server_base.cc:1034] running on GCE node
I20260808 10:49:08.707582 828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:08.708801 828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:08.709972 828 hybrid_clock.cc:648] HybridClock initialized: now 1786186148709936 us; error 49 us; skew 500 ppm
Aug 08 10:49:08 dist-test-slave-twwt krb5kdc[810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186148, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:08.713038 828 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:49:08.713949 828 webserver.cc:460] Webserver started at http://127.0.0.1:44625/ using document root <none> and password file <none>
I20260808 10:49:08.714511 828 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:08.714570 828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:08.714825 828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:08.716921 828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "6f45fa7a485c451a880f8e58a0b149cc"
format_stamp: "Formatted at 2026-08-08 10:49:08 on dist-test-slave-twwt"
server_key: "89cb7cad102b144de7cd33516577ca4c"
server_key_iv: "7e34057ef0835bfcfe6fa35d22588a08"
server_key_version: "encryptionkey@0"
I20260808 10:49:08.717448 828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "6f45fa7a485c451a880f8e58a0b149cc"
format_stamp: "Formatted at 2026-08-08 10:49:08 on dist-test-slave-twwt"
server_key: "89cb7cad102b144de7cd33516577ca4c"
server_key_iv: "7e34057ef0835bfcfe6fa35d22588a08"
server_key_version: "encryptionkey@0"
I20260808 10:49:08.721308 828 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 10:49:08.723786 856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:08.725226 828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:49:08.725384 828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "6f45fa7a485c451a880f8e58a0b149cc"
format_stamp: "Formatted at 2026-08-08 10:49:08 on dist-test-slave-twwt"
server_key: "89cb7cad102b144de7cd33516577ca4c"
server_key_iv: "7e34057ef0835bfcfe6fa35d22588a08"
server_key_version: "encryptionkey@0"
I20260808 10:49:08.725503 828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:08.739938 828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:08.743741 828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:08.743997 828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:08.751993 828 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34323
I20260808 10:49:08.752017 929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34323 every 8 connection(s)
I20260808 10:49:08.753149 828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:08.756270 930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:08.761112 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 828
I20260808 10:49:08.761245 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:08.761595 32694 external_mini_cluster.cc:1428] Setting key a3e156873a013e67cde7197b4f5de066
I20260808 10:49:08.763046 930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc: Bootstrap starting.
I20260808 10:49:08.766085 930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:08.767112 930 log.cc:824] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:08.769572 930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc: No bootstrap required, opened a new log
I20260808 10:49:08.773051 938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:08.763370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:35835 (local address 127.0.0.1:34323)
0808 10:49:08.763846 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:08.763866 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:08.763913 (+ 47us) server_negotiation.cc:407] Connection header received
0808 10:49:08.764875 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:08.764902 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:08.765393 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:08.765798 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:08.766298 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:08.767806 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:08.768828 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:08.769528 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:08.769712 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:08.770734 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:08.770740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:08.770757 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:08.771309 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:08.771382 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:08.771414 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:08.771560 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:08.771753 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 10:49:08.771904 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":133,"threads_started":1}
I20260808 10:49:08.774545 930 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f45fa7a485c451a880f8e58a0b149cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } }
I20260808 10:49:08.774854 930 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:49:08.774935 930 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f45fa7a485c451a880f8e58a0b149cc, State: Initialized, Role: FOLLOWER
I20260808 10:49:08.775482 930 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f45fa7a485c451a880f8e58a0b149cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } }
I20260808 10:49:08.775669 930 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:49:08.775740 930 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:49:08.775849 930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:08.776611 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:08.777087 930 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f45fa7a485c451a880f8e58a0b149cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } }
I20260808 10:49:08.777480 930 leader_election.cc:302] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f45fa7a485c451a880f8e58a0b149cc; no voters:
I20260808 10:49:08.777894 930 leader_election.cc:288] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:49:08.778009 946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:08.778252 946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [term 1 LEADER]: Becoming Leader. State: Replica: 6f45fa7a485c451a880f8e58a0b149cc, State: Running, Role: LEADER
I20260808 10:49:08.778647 946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f45fa7a485c451a880f8e58a0b149cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } }
I20260808 10:49:08.779529 930 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:08.780049 946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f45fa7a485c451a880f8e58a0b149cc. Latest consensus state: current_term: 1 leader_uuid: "6f45fa7a485c451a880f8e58a0b149cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f45fa7a485c451a880f8e58a0b149cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } } }
I20260808 10:49:08.780179 946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:08.780503 954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f45fa7a485c451a880f8e58a0b149cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f45fa7a485c451a880f8e58a0b149cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } } }
I20260808 10:49:08.780619 954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:08.784184 938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:08.779276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:38679 (local address 127.0.0.1:34323)
0808 10:49:08.779395 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:08.779400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:08.779415 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:49:08.779456 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:08.779459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:08.779508 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:08.779599 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:08.780006 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:08.780864 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:08.782052 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:08.782380 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:08.782457 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:08.783666 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:08.783670 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:08.783673 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:08.783688 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:08.783718 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:08.783731 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:08.783822 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:08.784003 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 10:49:08.784043 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":39}
W20260808 10:49:08.785367 978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:49:08.785454 978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:49:08.785590 959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:08.786425 959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:08.793112 959 catalog_manager.cc:1347] Generated new cluster ID: 1dcfe0f5b283423c939595ddf178c772
I20260808 10:49:08.793213 959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:08.801103 959 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:08.802398 959 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:08.811373 959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f45fa7a485c451a880f8e58a0b149cc: Generated new TSK 0
I20260808 10:49:08.812271 959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:49:08.823051 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 828
W20260808 10:49:08.839428 807 connection.cc:535] client connection to 127.0.0.1:34323 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34323): Connection reset by peer (error 104)
2026-08-08T10:49:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1770 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:49:08 dist-test-slave-twwt krb5kdc[987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:49:08 dist-test-slave-twwt krb5kdc[987](info): set up 2 sockets
Aug 08 10:49:08 dist-test-slave-twwt krb5kdc[987](info): commencing operation
krb5kdc: starting...
W20260808 10:49:10.118374 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:10 dist-test-slave-twwt krb5kdc[987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:49:10.253836 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35057
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:10.371413 1001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:10.371681 1001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:10.371726 1001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:10.375717 1001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:10.375782 1001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:10.375802 1001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:10.375818 1001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:10.375835 1001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:10.381129 1001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35057
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:10.382166 1001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:10.383268 1001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:10.392222 1009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:10.392313 1001 server_base.cc:1034] running on GCE node
W20260808 10:49:10.392251 1014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:10.392227 1010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:10.393206 1001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:10.394420 1001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:10.395642 1001 hybrid_clock.cc:648] HybridClock initialized: now 1786186150395626 us; error 34 us; skew 500 ppm
Aug 08 10:49:10 dist-test-slave-twwt krb5kdc[987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186150, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:10.405047 1001 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:49:10.406042 1001 webserver.cc:460] Webserver started at http://127.0.0.1:38671/ using document root <none> and password file <none>
I20260808 10:49:10.406697 1001 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:10.406780 1001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:10.406973 1001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:10.409184 1001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "1060ed70a4274565adc73d2f96645427"
format_stamp: "Formatted at 2026-08-08 10:49:10 on dist-test-slave-twwt"
server_key: "52dd01c3b104f0c6f9891fb0e45cf00b"
server_key_iv: "4e9e7f7adb9dc2b55c1647156ab812f0"
server_key_version: "encryptionkey@0"
I20260808 10:49:10.409757 1001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "1060ed70a4274565adc73d2f96645427"
format_stamp: "Formatted at 2026-08-08 10:49:10 on dist-test-slave-twwt"
server_key: "52dd01c3b104f0c6f9891fb0e45cf00b"
server_key_iv: "4e9e7f7adb9dc2b55c1647156ab812f0"
server_key_version: "encryptionkey@0"
I20260808 10:49:10.413954 1001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 10:49:10.416633 1028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:10.418017 1001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 10:49:10.418133 1001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "1060ed70a4274565adc73d2f96645427"
format_stamp: "Formatted at 2026-08-08 10:49:10 on dist-test-slave-twwt"
server_key: "52dd01c3b104f0c6f9891fb0e45cf00b"
server_key_iv: "4e9e7f7adb9dc2b55c1647156ab812f0"
server_key_version: "encryptionkey@0"
I20260808 10:49:10.418227 1001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:10.422880 1001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:10.426257 1001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:10.426419 1001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:10.434114 1001 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35057
I20260808 10:49:10.434142 1108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35057 every 8 connection(s)
I20260808 10:49:10.435182 1001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:10.438079 1109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:10.441818 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 1001
I20260808 10:49:10.441970 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:10.442312 32694 external_mini_cluster.cc:1428] Setting key 78f72be99b2edaecd3a3359ace76da21
I20260808 10:49:10.444550 1109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427: Bootstrap starting.
I20260808 10:49:10.447067 1109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:10.447985 1109 log.cc:824] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:10.448346 1113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:10.443932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:44585 (local address 127.0.0.1:35057)
0808 10:49:10.444446 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:10.444462 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:10.444501 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:49:10.445283 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:10.445304 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:10.445621 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:10.446004 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:10.446944 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:10.446949 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:10.446963 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:10.446976 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:10.447036 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:10.447061 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:10.447194 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:10.447323 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 10:49:10.447510 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":142,"threads_started":1}
I20260808 10:49:10.450263 1109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427: No bootstrap required, opened a new log
I20260808 10:49:10.451706 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:10.455332 1109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1060ed70a4274565adc73d2f96645427" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35057 } }
I20260808 10:49:10.455803 1109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:49:10.455891 1109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1060ed70a4274565adc73d2f96645427, State: Initialized, Role: FOLLOWER
I20260808 10:49:10.455911 1113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:10.454163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.192:46729 (local address 127.0.0.1:35057)
0808 10:49:10.454293 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:10.454297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:10.454442 (+ 145us) server_negotiation.cc:407] Connection header received
0808 10:49:10.454524 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:10.454527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:10.454564 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:10.454699 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:10.455453 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:10.455457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:10.455460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:10.455474 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:10.455505 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:10.455518 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:10.455612 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:10.455700 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 10:49:10.455747 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 10:49:10.456420 1109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1060ed70a4274565adc73d2f96645427" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35057 } }
I20260808 10:49:10.456565 1109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:49:10.456656 1109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:49:10.456753 1109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:10.457726 1109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1060ed70a4274565adc73d2f96645427" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35057 } }
I20260808 10:49:10.458101 1109 leader_election.cc:302] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1060ed70a4274565adc73d2f96645427; no voters:
I20260808 10:49:10.458468 1109 leader_election.cc:288] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:49:10.458546 1122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:10.458837 1122 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [term 1 LEADER]: Becoming Leader. State: Replica: 1060ed70a4274565adc73d2f96645427, State: Running, Role: LEADER
I20260808 10:49:10.459215 1122 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1060ed70a4274565adc73d2f96645427" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35057 } }
I20260808 10:49:10.459795 1109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:10.460928 1123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1060ed70a4274565adc73d2f96645427" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1060ed70a4274565adc73d2f96645427" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35057 } } }
I20260808 10:49:10.461102 1123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:10.461432 1124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1060ed70a4274565adc73d2f96645427. Latest consensus state: current_term: 1 leader_uuid: "1060ed70a4274565adc73d2f96645427" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1060ed70a4274565adc73d2f96645427" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35057 } } }
I20260808 10:49:10.461520 1124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:10.461536 1135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:10.464740 1135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:10.470755 1135 catalog_manager.cc:1347] Generated new cluster ID: 92e2284a8dee4e56bd1e3598034b8587
I20260808 10:49:10.470839 1135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:10.494864 1135 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:10.495270 1135 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:10.503850 1135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1060ed70a4274565adc73d2f96645427: Generated new TSK 0
I20260808 10:49:10.504604 1135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:49:10.514005 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 1001
2026-08-08T10:49:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1683 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:49:10 dist-test-slave-twwt krb5kdc[1155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:49:10 dist-test-slave-twwt krb5kdc[1155](info): set up 2 sockets
Aug 08 10:49:10 dist-test-slave-twwt krb5kdc[1155](info): commencing operation
krb5kdc: starting...
W20260808 10:49:11.754994 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:11 dist-test-slave-twwt krb5kdc[1155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:49:11.887626 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:38659
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:35951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:38659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:12.012467 1173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:12.012807 1173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:12.012895 1173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:12.016917 1173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:12.017024 1173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:12.017064 1173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:12.017124 1173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:12.017164 1173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:12.023166 1173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:35951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:38659
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:38659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:12.024569 1173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:12.025864 1173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:12.035041 1181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:12.035041 1185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:12.035041 1182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:12.035713 1173 server_base.cc:1034] running on GCE node
I20260808 10:49:12.036367 1173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:12.037557 1173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:12.038749 1173 hybrid_clock.cc:648] HybridClock initialized: now 1786186152038729 us; error 33 us; skew 500 ppm
Aug 08 10:49:12 dist-test-slave-twwt krb5kdc[1155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186152, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:12.041725 1173 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:49:12.042801 1173 webserver.cc:460] Webserver started at http://127.31.237.190:39757/ using document root <none> and password file <none>
I20260808 10:49:12.043406 1173 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:12.043499 1173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:12.043746 1173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:12.045890 1173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "49a3823bb5c74e15ad4a7d7c451ebc57"
format_stamp: "Formatted at 2026-08-08 10:49:12 on dist-test-slave-twwt"
server_key: "fbd2c8ed3785c3db8e351e76d23f9742"
server_key_iv: "155834b306d9e7a705df090ec8795882"
server_key_version: "encryptionkey@0"
I20260808 10:49:12.046486 1173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "49a3823bb5c74e15ad4a7d7c451ebc57"
format_stamp: "Formatted at 2026-08-08 10:49:12 on dist-test-slave-twwt"
server_key: "fbd2c8ed3785c3db8e351e76d23f9742"
server_key_iv: "155834b306d9e7a705df090ec8795882"
server_key_version: "encryptionkey@0"
I20260808 10:49:12.050341 1173 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 10:49:12.052973 1192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:12.054292 1173 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:49:12.054445 1173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "49a3823bb5c74e15ad4a7d7c451ebc57"
format_stamp: "Formatted at 2026-08-08 10:49:12 on dist-test-slave-twwt"
server_key: "fbd2c8ed3785c3db8e351e76d23f9742"
server_key_iv: "155834b306d9e7a705df090ec8795882"
server_key_version: "encryptionkey@0"
I20260808 10:49:12.054570 1173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:12.073836 1173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:12.077281 1173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:12.077499 1173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:12.085460 1173 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:38659
I20260808 10:49:12.085489 1280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:38659 every 8 connection(s)
I20260808 10:49:12.086649 1173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:12.089715 1281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:12.095980 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 1173
I20260808 10:49:12.096141 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:12.096247 1281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57: Bootstrap starting.
I20260808 10:49:12.096498 32694 external_mini_cluster.cc:1428] Setting key d1f8e2c71dafe9f1a41f345cf815bd68
I20260808 10:49:12.099138 1281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:12.099922 1281 log.cc:824] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:12.101974 1281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57: No bootstrap required, opened a new log
Aug 08 10:49:12 dist-test-slave-twwt krb5kdc[1155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:49:12.105772 1281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a3823bb5c74e15ad4a7d7c451ebc57" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38659 } }
I20260808 10:49:12.106139 1281 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:49:12.106225 1281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a3823bb5c74e15ad4a7d7c451ebc57, State: Initialized, Role: FOLLOWER
I20260808 10:49:12.106797 1281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a3823bb5c74e15ad4a7d7c451ebc57" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38659 } }
I20260808 10:49:12.107018 1281 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:49:12.107103 1281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:49:12.107236 1281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:12.108577 1281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a3823bb5c74e15ad4a7d7c451ebc57" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38659 } }
I20260808 10:49:12.108986 1281 leader_election.cc:302] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49a3823bb5c74e15ad4a7d7c451ebc57; no voters:
I20260808 10:49:12.109378 1281 leader_election.cc:288] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:49:12.109508 1287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:12.109836 1287 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [term 1 LEADER]: Becoming Leader. State: Replica: 49a3823bb5c74e15ad4a7d7c451ebc57, State: Running, Role: LEADER
I20260808 10:49:12.110209 1287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a3823bb5c74e15ad4a7d7c451ebc57" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38659 } }
I20260808 10:49:12.110878 1281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:12.112243 1291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49a3823bb5c74e15ad4a7d7c451ebc57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a3823bb5c74e15ad4a7d7c451ebc57" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38659 } } }
I20260808 10:49:12.112375 1291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:12.112824 1303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:12.112780 1292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49a3823bb5c74e15ad4a7d7c451ebc57. Latest consensus state: current_term: 1 leader_uuid: "49a3823bb5c74e15ad4a7d7c451ebc57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a3823bb5c74e15ad4a7d7c451ebc57" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 38659 } } }
I20260808 10:49:12.112933 1292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:12.113564 1284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:12.098064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53763 (local address 127.31.237.190:38659)
0808 10:49:12.098540 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:12.098556 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:12.098593 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:49:12.099406 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:12.099427 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:12.099789 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:12.100144 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:12.101036 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:12.102054 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:12.102996 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:12.103477 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:12.103649 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:12.106724 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:12.106731 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:12.106749 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:12.106783 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:12.108998 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:12.109639 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:12.109648 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:12.109655 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:12.109800 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:12.110241 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:12.110244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:12.110246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:12.110827 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:12.111661 (+ 834us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:12.111873 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 10:49:12.112152 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":157,"threads_started":1}
I20260808 10:49:12.116057 1303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:12.116995 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:12.122345 1303 catalog_manager.cc:1347] Generated new cluster ID: 5c77c88ab6ff439f86bc32ee6f242b80
I20260808 10:49:12.122409 1303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:12.128621 1284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:12.119366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40607 (local address 127.31.237.190:38659)
0808 10:49:12.119476 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:12.119482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:12.119734 (+ 252us) server_negotiation.cc:407] Connection header received
0808 10:49:12.119862 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:12.119865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:12.119916 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:12.120006 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:12.120836 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:12.121595 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:12.122816 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:12.123138 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:12.123291 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:12.125001 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:12.125005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:12.125008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:12.125034 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:12.126685 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:12.127262 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:12.127265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:12.127267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:12.127352 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:12.127713 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:12.127715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:12.127717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:12.127918 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:12.128059 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:12.128358 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 10:49:12.128457 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":40}
I20260808 10:49:12.144376 1303 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:12.145730 1303 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:12.163478 1303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49a3823bb5c74e15ad4a7d7c451ebc57: Generated new TSK 0
I20260808 10:49:12.164470 1303 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:49:12.170071 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 1173
W20260808 10:49:12.181689 1149 connection.cc:535] client connection to 127.31.237.190:38659 recv error: Network error: failed to read from TLS socket (remote: 127.31.237.190:38659): Connection reset by peer (error 104)
2026-08-08T10:49:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1658 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:49:12 dist-test-slave-twwt krb5kdc[1329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:49:12 dist-test-slave-twwt krb5kdc[1329](info): set up 2 sockets
Aug 08 10:49:12 dist-test-slave-twwt krb5kdc[1329](info): commencing operation
krb5kdc: starting...
W20260808 10:49:13.428028 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:13 dist-test-slave-twwt krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:49:13.564141 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:39815
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:33503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:39815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:13.679206 1352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:13.679477 1352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:13.679519 1352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:13.683509 1352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:13.683576 1352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:13.683595 1352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:13.683612 1352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:13.683627 1352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:13.688956 1352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:33503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:39815
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:39815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:13.690002 1352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:13.691084 1352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:13.700279 1360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:13.700398 1352 server_base.cc:1034] running on GCE node
W20260808 10:49:13.700279 1368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:13.700279 1366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:13.701123 1352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:13.702322 1352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:13.703498 1352 hybrid_clock.cc:648] HybridClock initialized: now 1786186153703464 us; error 46 us; skew 500 ppm
Aug 08 10:49:13 dist-test-slave-twwt krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186153, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:13.706326 1352 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:49:13.707261 1352 webserver.cc:460] Webserver started at http://127.31.237.190:41591/ using document root <none> and password file <none>
I20260808 10:49:13.707803 1352 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:13.707863 1352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:13.708032 1352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:13.710042 1352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "236dec49987d4a97b57e79864d57f9b4"
format_stamp: "Formatted at 2026-08-08 10:49:13 on dist-test-slave-twwt"
server_key: "9424ef0014695616d7d53fd3223202cb"
server_key_iv: "7bce80acde158de35d19dc658dc90ef0"
server_key_version: "encryptionkey@0"
I20260808 10:49:13.710548 1352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "236dec49987d4a97b57e79864d57f9b4"
format_stamp: "Formatted at 2026-08-08 10:49:13 on dist-test-slave-twwt"
server_key: "9424ef0014695616d7d53fd3223202cb"
server_key_iv: "7bce80acde158de35d19dc658dc90ef0"
server_key_version: "encryptionkey@0"
I20260808 10:49:13.714285 1352 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:49:13.716701 1377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:13.718014 1352 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:49:13.718122 1352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "236dec49987d4a97b57e79864d57f9b4"
format_stamp: "Formatted at 2026-08-08 10:49:13 on dist-test-slave-twwt"
server_key: "9424ef0014695616d7d53fd3223202cb"
server_key_iv: "7bce80acde158de35d19dc658dc90ef0"
server_key_version: "encryptionkey@0"
I20260808 10:49:13.718214 1352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:13.728787 1352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:13.732211 1352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:13.732370 1352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:13.740226 1352 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:39815
I20260808 10:49:13.740252 1453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:39815 every 8 connection(s)
I20260808 10:49:13.741268 1352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:13.741626 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 1352
I20260808 10:49:13.741786 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:13.742138 32694 external_mini_cluster.cc:1428] Setting key be0ec52a3e437c3cfdff15f9081828e1
I20260808 10:49:13.745008 1454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:13.752146 1454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4: Bootstrap starting.
I20260808 10:49:13.753393 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:13.743922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47741 (local address 127.31.237.190:39815)
0808 10:49:13.744513 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:13.744533 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:13.744625 (+ 92us) server_negotiation.cc:407] Connection header received
0808 10:49:13.745573 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:13.745597 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:13.746044 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:13.746416 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:13.746882 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:13.748398 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:13.749333 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:13.750054 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:13.750225 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:13.751314 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:13.751321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:13.751338 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:13.751847 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:13.751916 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:13.751945 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:13.752080 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:13.752299 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 10:49:13.752440 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":173,"threads_started":1}
I20260808 10:49:13.754952 1454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:13.755843 1454 log.cc:824] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:13.756672 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:13.758016 1454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4: No bootstrap required, opened a new log
I20260808 10:49:13.762869 1454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236dec49987d4a97b57e79864d57f9b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39815 } }
I20260808 10:49:13.763232 1454 raft_consensus.cc:374] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:49:13.763310 1454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 236dec49987d4a97b57e79864d57f9b4, State: Initialized, Role: FOLLOWER
I20260808 10:49:13.763597 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:13.758908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48599 (local address 127.31.237.190:39815)
0808 10:49:13.759019 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:13.759024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:13.759178 (+ 154us) server_negotiation.cc:407] Connection header received
0808 10:49:13.759274 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:13.759277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:13.759322 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:13.759400 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:13.759617 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:13.760467 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:13.761496 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:13.761843 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:13.761958 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:13.762944 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:13.762948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:13.762950 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:13.762965 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:13.762993 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:13.763006 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:13.763120 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:13.763397 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 10:49:13.763437 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:49:13.763852 1454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236dec49987d4a97b57e79864d57f9b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39815 } }
I20260808 10:49:13.764014 1454 raft_consensus.cc:388] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:49:13.764091 1454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:49:13.764187 1454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:13.765249 1454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236dec49987d4a97b57e79864d57f9b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39815 } }
I20260808 10:49:13.765566 1454 leader_election.cc:302] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 236dec49987d4a97b57e79864d57f9b4; no voters:
I20260808 10:49:13.765832 1454 leader_election.cc:288] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:49:13.765976 1469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:13.766268 1469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [term 1 LEADER]: Becoming Leader. State: Replica: 236dec49987d4a97b57e79864d57f9b4, State: Running, Role: LEADER
I20260808 10:49:13.766717 1469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236dec49987d4a97b57e79864d57f9b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39815 } }
I20260808 10:49:13.767258 1454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:13.768507 1472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 236dec49987d4a97b57e79864d57f9b4. Latest consensus state: current_term: 1 leader_uuid: "236dec49987d4a97b57e79864d57f9b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236dec49987d4a97b57e79864d57f9b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39815 } } }
I20260808 10:49:13.768649 1472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:13.769241 1485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:13.769474 1470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "236dec49987d4a97b57e79864d57f9b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236dec49987d4a97b57e79864d57f9b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 39815 } } }
I20260808 10:49:13.770390 1470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:13.772985 1485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:13.780009 1485 catalog_manager.cc:1347] Generated new cluster ID: 767599f3f4db4ef096fe5c14bfd4fa8b
I20260808 10:49:13.780148 1485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:13.796299 1485 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:13.797343 1485 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:13.804369 1485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 236dec49987d4a97b57e79864d57f9b4: Generated new TSK 0
I20260808 10:49:13.805111 1485 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:49:13.809510 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 1352
W20260808 10:49:13.823100 1327 connection.cc:535] client connection to 127.31.237.190:39815 recv error: Network error: failed to read from TLS socket (remote: 127.31.237.190:39815): Connection reset by peer (error 104)
2026-08-08T10:49:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1641 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:49:13 dist-test-slave-twwt krb5kdc[1498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:49:13 dist-test-slave-twwt krb5kdc[1498](info): set up 2 sockets
Aug 08 10:49:13 dist-test-slave-twwt krb5kdc[1498](info): commencing operation
krb5kdc: starting...
W20260808 10:49:15.069097 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:15 dist-test-slave-twwt krb5kdc[1498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:49:15.210422 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:36455
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:37801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:36455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:15.333155 1516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:15.333559 1516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:15.333660 1516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:15.337800 1516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:15.337913 1516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:15.337952 1516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:15.337988 1516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:15.338024 1516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:15.343581 1516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:37801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:36455
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:36455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:15.344769 1516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:15.345903 1516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:15.355171 1528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:15.355163 1530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:15.355163 1527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:15.355504 1516 server_base.cc:1034] running on GCE node
I20260808 10:49:15.356248 1516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:15.357566 1516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:15.358776 1516 hybrid_clock.cc:648] HybridClock initialized: now 1786186155358768 us; error 35 us; skew 500 ppm
Aug 08 10:49:15 dist-test-slave-twwt krb5kdc[1498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186155, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:15.361994 1516 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:49:15.363078 1516 webserver.cc:460] Webserver started at http://127.31.237.190:40537/ using document root <none> and password file <none>
I20260808 10:49:15.363689 1516 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:15.363785 1516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:15.364046 1516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:15.366429 1516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "c666b3da8d42446eb11988d4dee6b2b4"
format_stamp: "Formatted at 2026-08-08 10:49:15 on dist-test-slave-twwt"
server_key: "5ceb77e8a239644b569044d55bcae926"
server_key_iv: "d6be5ff71714291f16d2e395de9d8859"
server_key_version: "encryptionkey@0"
I20260808 10:49:15.367094 1516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "c666b3da8d42446eb11988d4dee6b2b4"
format_stamp: "Formatted at 2026-08-08 10:49:15 on dist-test-slave-twwt"
server_key: "5ceb77e8a239644b569044d55bcae926"
server_key_iv: "d6be5ff71714291f16d2e395de9d8859"
server_key_version: "encryptionkey@0"
I20260808 10:49:15.371174 1516 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:49:15.373761 1543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:15.375087 1516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:49:15.375239 1516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "c666b3da8d42446eb11988d4dee6b2b4"
format_stamp: "Formatted at 2026-08-08 10:49:15 on dist-test-slave-twwt"
server_key: "5ceb77e8a239644b569044d55bcae926"
server_key_iv: "d6be5ff71714291f16d2e395de9d8859"
server_key_version: "encryptionkey@0"
I20260808 10:49:15.375362 1516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:15.380169 1516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:15.383368 1516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:15.383566 1516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:15.391208 1516 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:36455
I20260808 10:49:15.391237 1621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:36455 every 8 connection(s)
I20260808 10:49:15.392179 1516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:15.395304 1622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:15.399096 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 1516
I20260808 10:49:15.399223 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:15.399537 32694 external_mini_cluster.cc:1428] Setting key 76c15dc288134e617cba6eff71e0c30c
I20260808 10:49:15.402256 1622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4: Bootstrap starting.
I20260808 10:49:15.405088 1622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:15.405861 1622 log.cc:824] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:15.407912 1622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4: No bootstrap required, opened a new log
I20260808 10:49:15.410096 1628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:15.401092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45915 (local address 127.31.237.190:36455)
0808 10:49:15.401568 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:15.401589 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:15.401636 (+ 47us) server_negotiation.cc:407] Connection header received
0808 10:49:15.402563 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:15.402590 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:15.403013 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:15.403424 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:15.408520 (+ 5096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:15.408526 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:15.408542 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:15.408557 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:15.408628 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:15.408655 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:15.408775 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:15.409004 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 10:49:15.409162 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":120,"threads_started":1}
I20260808 10:49:15.412483 1622 raft_consensus.cc:348] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c666b3da8d42446eb11988d4dee6b2b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36455 } }
I20260808 10:49:15.412792 1622 raft_consensus.cc:374] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:49:15.412885 1622 raft_consensus.cc:729] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c666b3da8d42446eb11988d4dee6b2b4, State: Initialized, Role: FOLLOWER
I20260808 10:49:15.413371 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:15.413384 1622 consensus_queue.cc:260] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c666b3da8d42446eb11988d4dee6b2b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36455 } }
I20260808 10:49:15.413552 1622 raft_consensus.cc:388] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:49:15.413618 1622 raft_consensus.cc:482] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:49:15.413723 1622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:15.414708 1622 raft_consensus.cc:504] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c666b3da8d42446eb11988d4dee6b2b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36455 } }
I20260808 10:49:15.415139 1622 leader_election.cc:302] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c666b3da8d42446eb11988d4dee6b2b4; no voters:
I20260808 10:49:15.415452 1622 leader_election.cc:288] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:49:15.415563 1637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:15.415889 1637 raft_consensus.cc:686] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [term 1 LEADER]: Becoming Leader. State: Replica: c666b3da8d42446eb11988d4dee6b2b4, State: Running, Role: LEADER
I20260808 10:49:15.416296 1637 consensus_queue.cc:237] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c666b3da8d42446eb11988d4dee6b2b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36455 } }
I20260808 10:49:15.416791 1622 sys_catalog.cc:564] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:15.417984 1628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:15.415781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52293 (local address 127.31.237.190:36455)
0808 10:49:15.415882 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:15.415887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:15.416441 (+ 554us) server_negotiation.cc:407] Connection header received
0808 10:49:15.416573 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:15.416576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:15.416612 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:15.416697 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:15.417492 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:15.417496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:15.417498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:15.417511 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:15.417541 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:15.417558 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:15.417658 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:15.417797 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 10:49:15.417845 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:49:15.418426 1639 sys_catalog.cc:455] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c666b3da8d42446eb11988d4dee6b2b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c666b3da8d42446eb11988d4dee6b2b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36455 } } }
I20260808 10:49:15.418566 1639 sys_catalog.cc:458] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:15.419436 1658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:15.419204 1641 sys_catalog.cc:455] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c666b3da8d42446eb11988d4dee6b2b4. Latest consensus state: current_term: 1 leader_uuid: "c666b3da8d42446eb11988d4dee6b2b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c666b3da8d42446eb11988d4dee6b2b4" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 36455 } } }
I20260808 10:49:15.419521 1641 sys_catalog.cc:458] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:15.422178 1658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:15.427533 1658 catalog_manager.cc:1347] Generated new cluster ID: 36732afb6aeb4f0f90c286749238545e
I20260808 10:49:15.427600 1658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:15.439723 1658 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:15.440097 1658 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:15.449640 1658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c666b3da8d42446eb11988d4dee6b2b4: Generated new TSK 0
I20260808 10:49:15.450409 1658 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:49:15.463272 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 1516
2026-08-08T10:49:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1659 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:49:15 dist-test-slave-twwt krb5kdc[1670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:49:15 dist-test-slave-twwt krb5kdc[1670](info): set up 2 sockets
Aug 08 10:49:15 dist-test-slave-twwt krb5kdc[1670](info): commencing operation
krb5kdc: starting...
W20260808 10:49:16.750151 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:16 dist-test-slave-twwt krb5kdc[1670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:49:16.881702 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:43167
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:43167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:16.996380 1690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:16.996697 1690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:16.996801 1690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:17.000964 1690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:17.001060 1690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:17.001099 1690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:17.001151 1690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:17.001187 1690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:17.006479 1690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:43167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:43167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:17.007649 1690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:17.008805 1690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:17.017711 1699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:17.017684 1698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:17.017771 1690 server_base.cc:1034] running on GCE node
W20260808 10:49:17.017704 1701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:17.018751 1690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:17.019959 1690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:17.021155 1690 hybrid_clock.cc:648] HybridClock initialized: now 1786186157021139 us; error 34 us; skew 500 ppm
Aug 08 10:49:17 dist-test-slave-twwt krb5kdc[1670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186157, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:17.024209 1690 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:49:17.025203 1690 webserver.cc:460] Webserver started at http://127.31.237.190:33473/ using document root <none> and password file <none>
I20260808 10:49:17.025799 1690 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:17.025902 1690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:17.026149 1690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:17.028313 1690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "35e8c6f34ba94cb3b1e26677279a26ec"
format_stamp: "Formatted at 2026-08-08 10:49:17 on dist-test-slave-twwt"
server_key: "7e5fc91a762bbe369ec5a886a1b1728b"
server_key_iv: "7ba435ed70eb59e239ed393223557a3d"
server_key_version: "encryptionkey@0"
I20260808 10:49:17.028918 1690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "35e8c6f34ba94cb3b1e26677279a26ec"
format_stamp: "Formatted at 2026-08-08 10:49:17 on dist-test-slave-twwt"
server_key: "7e5fc91a762bbe369ec5a886a1b1728b"
server_key_iv: "7ba435ed70eb59e239ed393223557a3d"
server_key_version: "encryptionkey@0"
I20260808 10:49:17.033054 1690 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:49:17.035686 1709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:17.037051 1690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:49:17.037204 1690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "35e8c6f34ba94cb3b1e26677279a26ec"
format_stamp: "Formatted at 2026-08-08 10:49:17 on dist-test-slave-twwt"
server_key: "7e5fc91a762bbe369ec5a886a1b1728b"
server_key_iv: "7ba435ed70eb59e239ed393223557a3d"
server_key_version: "encryptionkey@0"
I20260808 10:49:17.037329 1690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:17.054829 1690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:17.058036 1690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:17.058259 1690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:17.065788 1690 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:43167
I20260808 10:49:17.065798 1801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:43167 every 8 connection(s)
I20260808 10:49:17.066816 1690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:17.069407 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 1690
I20260808 10:49:17.069573 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:17.069967 32694 external_mini_cluster.cc:1428] Setting key 5475e3305c01941cb4ef82ac8b9b58a1
I20260808 10:49:17.070133 1802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:17.077913 1802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec: Bootstrap starting.
Aug 08 10:49:17 dist-test-slave-twwt krb5kdc[1670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:49:17.080829 1802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:17.081823 1802 log.cc:824] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:17.084245 1802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec: No bootstrap required, opened a new log
I20260808 10:49:17.085752 1805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:17.071454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35827 (local address 127.31.237.190:43167)
0808 10:49:17.071937 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:17.071963 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:17.072000 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:49:17.072662 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:17.072679 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:17.073041 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:17.073324 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:17.074271 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:17.075374 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:17.076617 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:17.077112 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:17.077247 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:17.080328 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:17.080339 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:17.080355 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:17.080386 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:17.082491 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:17.083033 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:17.083039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:17.083044 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:17.083182 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:17.083462 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:17.083465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:17.083467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:17.083994 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:17.084186 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:17.084478 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 10:49:17.084799 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":146,"threads_started":1}
I20260808 10:49:17.088280 1802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35e8c6f34ba94cb3b1e26677279a26ec" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43167 } }
I20260808 10:49:17.088546 1802 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:49:17.088611 1802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35e8c6f34ba94cb3b1e26677279a26ec, State: Initialized, Role: FOLLOWER
I20260808 10:49:17.089133 1802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35e8c6f34ba94cb3b1e26677279a26ec" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43167 } }
I20260808 10:49:17.089263 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:17.089285 1802 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:49:17.089381 1802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:49:17.089498 1802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:17.090498 1802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35e8c6f34ba94cb3b1e26677279a26ec" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43167 } }
I20260808 10:49:17.090919 1802 leader_election.cc:302] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35e8c6f34ba94cb3b1e26677279a26ec; no voters:
I20260808 10:49:17.091251 1802 leader_election.cc:288] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:49:17.091329 1809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:17.091583 1809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [term 1 LEADER]: Becoming Leader. State: Replica: 35e8c6f34ba94cb3b1e26677279a26ec, State: Running, Role: LEADER
I20260808 10:49:17.091945 1809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35e8c6f34ba94cb3b1e26677279a26ec" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43167 } }
I20260808 10:49:17.092697 1802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:17.093973 1810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35e8c6f34ba94cb3b1e26677279a26ec. Latest consensus state: current_term: 1 leader_uuid: "35e8c6f34ba94cb3b1e26677279a26ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35e8c6f34ba94cb3b1e26677279a26ec" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43167 } } }
I20260808 10:49:17.094009 1811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35e8c6f34ba94cb3b1e26677279a26ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35e8c6f34ba94cb3b1e26677279a26ec" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 43167 } } }
I20260808 10:49:17.094120 1810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:17.094130 1811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:17.094553 1823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:17.097950 1823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:17.104228 1805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:17.095410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48877 (local address 127.31.237.190:43167)
0808 10:49:17.095539 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:17.095544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:17.095881 (+ 337us) server_negotiation.cc:407] Connection header received
0808 10:49:17.096100 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:17.096103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:17.096155 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:17.096242 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:17.097068 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:17.097854 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:17.098996 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:17.099365 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:17.099481 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:17.101024 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:17.101028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:17.101031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:17.101057 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:17.102518 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:17.102964 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:17.102968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:17.102970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:17.103058 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:17.103371 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:17.103374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:17.103375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:17.103559 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:17.103708 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:17.103947 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 10:49:17.104062 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:49:17.105652 1823 catalog_manager.cc:1347] Generated new cluster ID: 9419945c3b83455192522d0ae757ad2f
I20260808 10:49:17.105755 1823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:17.122069 1823 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:17.123309 1823 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:17.135555 1823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35e8c6f34ba94cb3b1e26677279a26ec: Generated new TSK 0
I20260808 10:49:17.136472 1823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:49:17.149890 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 1690
2026-08-08T10:49:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1684 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:49:17 dist-test-slave-twwt krb5kdc[1841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:49:17 dist-test-slave-twwt krb5kdc[1841](info): set up 2 sockets
Aug 08 10:49:17 dist-test-slave-twwt krb5kdc[1841](info): commencing operation
krb5kdc: starting...
W20260808 10:49:18.409541 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:18 dist-test-slave-twwt krb5kdc[1841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:49:18.541851 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:34737
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:34737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:18.663247 1862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:18.663582 1862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:18.663676 1862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:18.667984 1862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:18.668073 1862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:18.668113 1862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:18.668174 1862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:18.668215 1862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:18.673614 1862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:34737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:34737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:18.674813 1862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:18.675870 1862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:18.684926 1873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:18.684918 1870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:18.684860 1876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:18.684893 1862 server_base.cc:1034] running on GCE node
I20260808 10:49:18.685631 1862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:18.686784 1862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:18.687951 1862 hybrid_clock.cc:648] HybridClock initialized: now 1786186158687931 us; error 31 us; skew 500 ppm
Aug 08 10:49:18 dist-test-slave-twwt krb5kdc[1841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186158, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:18.691502 1862 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:49:18.692431 1862 webserver.cc:460] Webserver started at http://127.31.237.190:44515/ using document root <none> and password file <none>
I20260808 10:49:18.693069 1862 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:18.693166 1862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:18.693413 1862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:18.695588 1862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "1c4620dde2ff4ad28b9e91e85bb30189"
format_stamp: "Formatted at 2026-08-08 10:49:18 on dist-test-slave-twwt"
server_key: "40ab37eee1a9d8dcfc2cf3a075807b71"
server_key_iv: "2e1d2a0c462647a13d6e644cfffe03bc"
server_key_version: "encryptionkey@0"
I20260808 10:49:18.696173 1862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "1c4620dde2ff4ad28b9e91e85bb30189"
format_stamp: "Formatted at 2026-08-08 10:49:18 on dist-test-slave-twwt"
server_key: "40ab37eee1a9d8dcfc2cf3a075807b71"
server_key_iv: "2e1d2a0c462647a13d6e644cfffe03bc"
server_key_version: "encryptionkey@0"
I20260808 10:49:18.700223 1862 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260808 10:49:18.703045 1888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:18.704499 1862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:49:18.704674 1862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "1c4620dde2ff4ad28b9e91e85bb30189"
format_stamp: "Formatted at 2026-08-08 10:49:18 on dist-test-slave-twwt"
server_key: "40ab37eee1a9d8dcfc2cf3a075807b71"
server_key_iv: "2e1d2a0c462647a13d6e644cfffe03bc"
server_key_version: "encryptionkey@0"
I20260808 10:49:18.704818 1862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:18.731755 1862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:18.735284 1862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:18.735502 1862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:18.743320 1862 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:34737
I20260808 10:49:18.743346 1979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:34737 every 8 connection(s)
I20260808 10:49:18.744310 1862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:18.747336 1981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:18.749028 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 1862
I20260808 10:49:18.749173 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:18.749475 32694 external_mini_cluster.cc:1428] Setting key 6a811dc4cb83f2f6d606d98a5faa515b
I20260808 10:49:18.754405 1981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189: Bootstrap starting.
I20260808 10:49:18.757118 1981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:18.758103 1981 log.cc:824] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:18.759716 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:18.750982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43273 (local address 127.31.237.190:34737)
0808 10:49:18.751436 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:18.751459 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:18.751505 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:49:18.752387 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:18.752413 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:18.752879 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:18.753262 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:18.753700 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:18.755169 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:18.756101 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:18.756770 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:18.756941 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:18.757764 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:18.757771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:18.757789 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:18.758313 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:18.758385 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:18.758417 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:18.758521 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:18.758726 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 10:49:18.758957 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":107,"threads_started":1}
I20260808 10:49:18.760277 1981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189: No bootstrap required, opened a new log
I20260808 10:49:18.762952 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:18.764923 1981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4620dde2ff4ad28b9e91e85bb30189" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34737 } }
I20260808 10:49:18.765223 1981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:49:18.765336 1981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c4620dde2ff4ad28b9e91e85bb30189, State: Initialized, Role: FOLLOWER
I20260808 10:49:18.765846 1981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4620dde2ff4ad28b9e91e85bb30189" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34737 } }
I20260808 10:49:18.765980 1981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:49:18.766090 1981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:49:18.766206 1981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:18.767293 1981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4620dde2ff4ad28b9e91e85bb30189" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34737 } }
I20260808 10:49:18.767656 1981 leader_election.cc:302] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c4620dde2ff4ad28b9e91e85bb30189; no voters:
I20260808 10:49:18.767967 1981 leader_election.cc:288] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:49:18.768064 1992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:18.768318 1992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [term 1 LEADER]: Becoming Leader. State: Replica: 1c4620dde2ff4ad28b9e91e85bb30189, State: Running, Role: LEADER
I20260808 10:49:18.768728 1992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4620dde2ff4ad28b9e91e85bb30189" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34737 } }
I20260808 10:49:18.769349 1981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:18.770046 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:18.765349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34935 (local address 127.31.237.190:34737)
0808 10:49:18.765490 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:18.765494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:18.765737 (+ 243us) server_negotiation.cc:407] Connection header received
0808 10:49:18.765921 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:18.765923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:18.765995 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:18.766092 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:18.766419 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:18.767015 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:18.767927 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:18.768317 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:18.768430 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:18.769388 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:18.769392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:18.769395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:18.769409 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:18.769440 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:18.769453 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:18.769527 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:18.769861 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 10:49:18.769919 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:49:18.771986 1996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c4620dde2ff4ad28b9e91e85bb30189" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4620dde2ff4ad28b9e91e85bb30189" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34737 } } }
I20260808 10:49:18.771586 1994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c4620dde2ff4ad28b9e91e85bb30189. Latest consensus state: current_term: 1 leader_uuid: "1c4620dde2ff4ad28b9e91e85bb30189" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4620dde2ff4ad28b9e91e85bb30189" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 34737 } } }
I20260808 10:49:18.772192 1996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:18.772266 1994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189 [sys.catalog]: This master's current role is: LEADER
W20260808 10:49:18.774386 2010 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:49:18.774479 2010 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:49:18.774673 2012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:18.775418 2012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:18.781083 2012 catalog_manager.cc:1347] Generated new cluster ID: c6c68a7b7021467ca30146c174c03f20
I20260808 10:49:18.781154 2012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:18.796692 2012 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:18.798034 2012 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:18.805588 2012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c4620dde2ff4ad28b9e91e85bb30189: Generated new TSK 0
I20260808 10:49:18.806355 2012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:49:18.819828 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 1862
2026-08-08T10:49:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1664 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:49:18 dist-test-slave-twwt krb5kdc[2017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:49:18 dist-test-slave-twwt krb5kdc[2017](info): set up 2 sockets
Aug 08 10:49:18 dist-test-slave-twwt krb5kdc[2017](info): commencing operation
krb5kdc: starting...
W20260808 10:49:20.087585 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:20 dist-test-slave-twwt krb5kdc[2017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:49:20.221143 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:42071
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:37293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:42071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:20.339430 2041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:20.339749 2041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:20.339851 2041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:20.343878 2041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:20.343971 2041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:20.344012 2041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:20.344049 2041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:20.344089 2041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:20.349484 2041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:37293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:42071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:42071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:20.350736 2041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:20.351852 2041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:20.361377 2056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:20.361349 2053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:20.361470 2041 server_base.cc:1034] running on GCE node
W20260808 10:49:20.361373 2051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:20.362252 2041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:20.363508 2041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:20.364710 2041 hybrid_clock.cc:648] HybridClock initialized: now 1786186160364677 us; error 43 us; skew 500 ppm
Aug 08 10:49:20 dist-test-slave-twwt krb5kdc[2017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186160, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:20.367738 2041 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:49:20.368695 2041 webserver.cc:460] Webserver started at http://127.31.237.190:38817/ using document root <none> and password file <none>
I20260808 10:49:20.369267 2041 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:20.369341 2041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:20.369529 2041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:20.371820 2041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "dae5ca4b237a44c990db3352ae9557f5"
format_stamp: "Formatted at 2026-08-08 10:49:20 on dist-test-slave-twwt"
server_key: "f6ea90f638694bf27b595537246536d2"
server_key_iv: "14f149172bbb06e3276207920399d57c"
server_key_version: "encryptionkey@0"
I20260808 10:49:20.372402 2041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "dae5ca4b237a44c990db3352ae9557f5"
format_stamp: "Formatted at 2026-08-08 10:49:20 on dist-test-slave-twwt"
server_key: "f6ea90f638694bf27b595537246536d2"
server_key_iv: "14f149172bbb06e3276207920399d57c"
server_key_version: "encryptionkey@0"
I20260808 10:49:20.376456 2041 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 10:49:20.379189 2070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:20.380511 2041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:49:20.380621 2041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "dae5ca4b237a44c990db3352ae9557f5"
format_stamp: "Formatted at 2026-08-08 10:49:20 on dist-test-slave-twwt"
server_key: "f6ea90f638694bf27b595537246536d2"
server_key_iv: "14f149172bbb06e3276207920399d57c"
server_key_version: "encryptionkey@0"
I20260808 10:49:20.380831 2041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:20.385958 2041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:20.389142 2041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:20.389413 2041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:20.396878 2041 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:42071
I20260808 10:49:20.396909 2147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:42071 every 8 connection(s)
I20260808 10:49:20.397850 2041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:20.398099 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 2041
I20260808 10:49:20.398272 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:20.398622 32694 external_mini_cluster.cc:1428] Setting key dcc0badc124361d851737f1d0e4f1cf8
I20260808 10:49:20.401163 2152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:20.404202 2156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:20.400142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53265 (local address 127.31.237.190:42071)
0808 10:49:20.400583 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:20.400600 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:20.400635 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:49:20.401357 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:20.401376 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:20.401669 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:20.401970 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:20.402951 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:20.402970 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:20.402983 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:20.402994 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:20.403051 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:20.403071 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:20.403187 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:20.403301 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 10:49:20.403473 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":148,"threads_started":1}
I20260808 10:49:20.407119 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:20.407701 2152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5: Bootstrap starting.
I20260808 10:49:20.410385 2152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:20.411255 2152 log.cc:824] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:20.413546 2152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5: No bootstrap required, opened a new log
I20260808 10:49:20.413702 2156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:20.409274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40823 (local address 127.31.237.190:42071)
0808 10:49:20.409415 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:20.409419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:20.409614 (+ 195us) server_negotiation.cc:407] Connection header received
0808 10:49:20.409735 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:20.409737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:20.409768 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:20.409833 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:20.413182 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:20.413186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:20.413188 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:49:20.413200 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:20.413226 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:49:20.413236 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:20.413320 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:20.413443 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 10:49:20.413479 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:49:20.417271 2152 raft_consensus.cc:348] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae5ca4b237a44c990db3352ae9557f5" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 42071 } }
I20260808 10:49:20.417546 2152 raft_consensus.cc:374] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:49:20.417609 2152 raft_consensus.cc:729] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dae5ca4b237a44c990db3352ae9557f5, State: Initialized, Role: FOLLOWER
I20260808 10:49:20.418066 2152 consensus_queue.cc:260] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae5ca4b237a44c990db3352ae9557f5" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 42071 } }
I20260808 10:49:20.418251 2152 raft_consensus.cc:388] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:49:20.418336 2152 raft_consensus.cc:482] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:49:20.418454 2152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:20.419502 2152 raft_consensus.cc:504] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae5ca4b237a44c990db3352ae9557f5" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 42071 } }
I20260808 10:49:20.419814 2152 leader_election.cc:302] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dae5ca4b237a44c990db3352ae9557f5; no voters:
I20260808 10:49:20.420086 2152 leader_election.cc:288] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:49:20.420213 2177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:20.420487 2177 raft_consensus.cc:686] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [term 1 LEADER]: Becoming Leader. State: Replica: dae5ca4b237a44c990db3352ae9557f5, State: Running, Role: LEADER
I20260808 10:49:20.420890 2177 consensus_queue.cc:237] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae5ca4b237a44c990db3352ae9557f5" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 42071 } }
I20260808 10:49:20.421398 2152 sys_catalog.cc:564] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:20.422658 2182 sys_catalog.cc:455] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dae5ca4b237a44c990db3352ae9557f5. Latest consensus state: current_term: 1 leader_uuid: "dae5ca4b237a44c990db3352ae9557f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae5ca4b237a44c990db3352ae9557f5" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 42071 } } }
I20260808 10:49:20.422758 2179 sys_catalog.cc:455] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dae5ca4b237a44c990db3352ae9557f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dae5ca4b237a44c990db3352ae9557f5" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 42071 } } }
I20260808 10:49:20.422941 2179 sys_catalog.cc:458] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:20.423223 2182 sys_catalog.cc:458] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5 [sys.catalog]: This master's current role is: LEADER
W20260808 10:49:20.426577 2201 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:49:20.426708 2201 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:49:20.426803 2197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:20.427508 2197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:20.433271 2197 catalog_manager.cc:1347] Generated new cluster ID: 99e8c5bd5480428f8f6dc5c92c5cf4d1
I20260808 10:49:20.433377 2197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:20.461463 2197 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:20.461825 2197 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:20.469007 2197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dae5ca4b237a44c990db3352ae9557f5: Generated new TSK 0
I20260808 10:49:20.469727 2197 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:49:20.478936 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 2041
2026-08-08T10:49:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1660 ms)
[----------] 18 tests from AuthTokenIssuingTest (30245 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:49:20 dist-test-slave-twwt krb5kdc[2212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:49:20 dist-test-slave-twwt krb5kdc[2212](info): set up 2 sockets
Aug 08 10:49:20 dist-test-slave-twwt krb5kdc[2212](info): commencing operation
krb5kdc: starting...
W20260808 10:49:21.777827 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:21 dist-test-slave-twwt krb5kdc[2212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:49:21.909245 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:45195
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:45195,127.31.237.189:42489,127.31.237.188:42365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:22.031666 2233 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:49:22.032086 2233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:22.032164 2233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:22.032250 2233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:22.036643 2233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:22.036742 2233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:22.036782 2233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:22.036849 2233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:22.036887 2233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:22.042531 2233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:45195,127.31.237.189:42489,127.31.237.188:42365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:45195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:22.043771 2233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:22.044885 2233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:22.054150 2240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:22.054150 2245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:22.054160 2239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:22.054399 2233 server_base.cc:1034] running on GCE node
I20260808 10:49:22.055047 2233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:22.056084 2233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:22.057279 2233 hybrid_clock.cc:648] HybridClock initialized: now 1786186162057259 us; error 31 us; skew 500 ppm
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:22.060096 2233 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:49:22.060977 2233 webserver.cc:460] Webserver started at http://127.31.237.190:42811/ using document root <none> and password file <none>
I20260808 10:49:22.061698 2233 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:22.061817 2233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:22.062089 2233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:22.064771 2233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "1c0da80e08914545b09a10390cde764f"
format_stamp: "Formatted at 2026-08-08 10:49:22 on dist-test-slave-twwt"
server_key: "b4558841a4153c1b65f73ffc481e62a2"
server_key_iv: "76baf4f349bc2dcb570cad3d71fa1954"
server_key_version: "encryptionkey@0"
I20260808 10:49:22.065392 2233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "1c0da80e08914545b09a10390cde764f"
format_stamp: "Formatted at 2026-08-08 10:49:22 on dist-test-slave-twwt"
server_key: "b4558841a4153c1b65f73ffc481e62a2"
server_key_iv: "76baf4f349bc2dcb570cad3d71fa1954"
server_key_version: "encryptionkey@0"
I20260808 10:49:22.069168 2233 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 10:49:22.072165 2262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:22.073714 2233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 10:49:22.073920 2233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "1c0da80e08914545b09a10390cde764f"
format_stamp: "Formatted at 2026-08-08 10:49:22 on dist-test-slave-twwt"
server_key: "b4558841a4153c1b65f73ffc481e62a2"
server_key_iv: "76baf4f349bc2dcb570cad3d71fa1954"
server_key_version: "encryptionkey@0"
I20260808 10:49:22.074060 2233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:22.098093 2233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:22.101425 2233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:22.101660 2233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:22.109736 2233 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:45195
I20260808 10:49:22.110036 2344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:45195 every 8 connection(s)
I20260808 10:49:22.110939 2233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:22.113870 2345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:22.116988 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 2233
I20260808 10:49:22.117146 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:22.117471 32694 external_mini_cluster.cc:1428] Setting key 9e7fa26b8e3f16314fdd15d662344888
I20260808 10:49:22.118283 2345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.237.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.189@KRBTEST.COM" created.
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:49:22.137593 2349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.121165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43772 (local address 127.31.237.190:45195)
0808 10:49:22.121783 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.121812 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.121868 (+ 56us) server_negotiation.cc:407] Connection header received
0808 10:49:22.123088 (+ 1220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.123098 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.123372 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.123594 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.124412 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.125926 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.127029 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.127670 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.127876 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.132264 (+ 4388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.132270 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.132283 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.132308 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.134141 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.134703 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.134712 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.134718 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.134868 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.135233 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.135239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.135244 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.135837 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.135997 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.136423 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.136665 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":481,"spinlock_wait_cycles":7424,"thread_start_us":171,"threads_started":1}
I20260808 10:49:22.137596 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.121165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:45195 (local address 127.0.0.1:43772)
0808 10:49:22.121711 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.121766 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:22.122726 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:22.123602 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:22.123607 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:22.123953 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:22.124244 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.124257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.126075 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.126078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.126900 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.126910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.127886 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.127890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.127986 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.128621 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:22.128639 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:22.132081 (+ 3442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:22.134279 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.134291 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.134302 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.134550 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.134985 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.134988 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.134990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.135119 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.135967 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:22.135973 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:22.136114 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 10:49:22.136260 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 10:49:22.136566 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":161,"threads_started":1}
I20260808 10:49:22.140722 2345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:49:22.141594 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.141232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:42489 (local address 127.0.0.1:38484)
0808 10:49:22.141407 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.141509 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 10:49:22.141829 2268 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.237.189:42489: connect: Connection refused (error 111)
I20260808 10:49:22.142738 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.142456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:42489 (local address 127.0.0.1:38490)
0808 10:49:22.142578 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.142685 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 10:49:22.142900 2345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.189:42489: Network error: Client connection negotiation failed: client connection to 127.31.237.189:42489: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.189.keytab.
Entry for principal kudu/127.31.237.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.189.keytab.
I20260808 10:49:22.149953 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.189:42489
--webserver_interface=127.31.237.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.189.keytab
--principal=kudu/127.31.237.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:45195,127.31.237.189:42489,127.31.237.188:42365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:49:22.192226 2345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } attempt: 1
I20260808 10:49:22.193231 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.192851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:42489 (local address 127.0.0.1:38496)
0808 10:49:22.193052 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.193132 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 10:49:22.194020 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.193766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:42489 (local address 127.0.0.1:38510)
0808 10:49:22.193875 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.193928 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 10:49:22.194183 2345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.189:42489: Network error: Client connection negotiation failed: client connection to 127.31.237.189:42489: connect: Connection refused (error 111)
I20260808 10:49:22.262486 2345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } attempt: 2
I20260808 10:49:22.263501 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.263107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:42489 (local address 127.0.0.1:38514)
0808 10:49:22.263316 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.263402 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 10:49:22.264429 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.264181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:42489 (local address 127.0.0.1:38528)
0808 10:49:22.264299 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.264357 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 10:49:22.264627 2345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.189:42489: Network error: Client connection negotiation failed: client connection to 127.31.237.189:42489: connect: Connection refused (error 111)
W20260808 10:49:22.268716 2352 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:49:22.268998 2352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:22.269069 2352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:22.269143 2352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:22.273370 2352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:22.273430 2352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:22.273450 2352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:22.273517 2352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:22.273551 2352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:22.279338 2352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:45195,127.31.237.189:42489,127.31.237.188:42365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.189.keytab
--principal=kudu/127.31.237.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.189:42489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.237.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:22.280485 2352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:22.281556 2352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:22.290639 2360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:22.290638 2363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:22.290691 2359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:22.290906 2352 server_base.cc:1034] running on GCE node
I20260808 10:49:22.291471 2352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:22.292683 2352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:22.293867 2352 hybrid_clock.cc:648] HybridClock initialized: now 1786186162293853 us; error 27 us; skew 500 ppm
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:22.296784 2352 init.cc:377] Logged in from keytab as kudu/127.31.237.189@KRBTEST.COM (short username kudu)
I20260808 10:49:22.297729 2352 webserver.cc:460] Webserver started at http://127.31.237.189:33719/ using document root <none> and password file <none>
I20260808 10:49:22.298273 2352 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:22.298332 2352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:22.298513 2352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:22.300627 2352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/data/instance:
uuid: "50621ca72fc24af3a3a1495756d5adbe"
format_stamp: "Formatted at 2026-08-08 10:49:22 on dist-test-slave-twwt"
server_key: "6a96c5e04b3d71840d2526152466fa66"
server_key_iv: "4234349b3e56199fbc4803aa249ef27c"
server_key_version: "encryptionkey@0"
I20260808 10:49:22.301221 2352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/wal/instance:
uuid: "50621ca72fc24af3a3a1495756d5adbe"
format_stamp: "Formatted at 2026-08-08 10:49:22 on dist-test-slave-twwt"
server_key: "6a96c5e04b3d71840d2526152466fa66"
server_key_iv: "4234349b3e56199fbc4803aa249ef27c"
server_key_version: "encryptionkey@0"
I20260808 10:49:22.305064 2352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:49:22.307591 2373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:22.308837 2352 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:49:22.309055 2352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/wal
uuid: "50621ca72fc24af3a3a1495756d5adbe"
format_stamp: "Formatted at 2026-08-08 10:49:22 on dist-test-slave-twwt"
server_key: "6a96c5e04b3d71840d2526152466fa66"
server_key_iv: "4234349b3e56199fbc4803aa249ef27c"
server_key_version: "encryptionkey@0"
I20260808 10:49:22.309175 2352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:22.352288 2352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:22.355829 2345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } attempt: 3
I20260808 10:49:22.355989 2352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:22.356218 2352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:22.356679 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.356409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:42489 (local address 127.0.0.1:56586)
0808 10:49:22.356544 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.356602 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 10:49:22.357553 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.357301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:42489 (local address 127.0.0.1:56602)
0808 10:49:22.357427 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.357482 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:42489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 10:49:22.357728 2345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.189:42489: Network error: Client connection negotiation failed: client connection to 127.31.237.189:42489: connect: Connection refused (error 111)
I20260808 10:49:22.365097 2352 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.189:42489
I20260808 10:49:22.365463 2454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.189:42489 every 8 connection(s)
I20260808 10:49:22.366658 2352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/data/info.pb
I20260808 10:49:22.367827 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 2352
I20260808 10:49:22.367971 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-1/wal/instance
I20260808 10:49:22.368263 32694 external_mini_cluster.cc:1428] Setting key 40bcefca61175bae270f0c3f0e4cd04c
I20260808 10:49:22.370967 2456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:22.375217 2456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.237.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.188@KRBTEST.COM" created.
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.189@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:49:22.390328 2349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.377223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44024 (local address 127.31.237.190:45195)
0808 10:49:22.377361 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.377364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.378469 (+ 1105us) server_negotiation.cc:407] Connection header received
0808 10:49:22.379540 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.379544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.379598 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.379676 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.380999 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.381542 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.382889 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.383134 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.383249 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.386169 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.386173 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.386175 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.386203 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.387897 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.388486 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.388491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.388493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.388585 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.388977 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.388984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.388987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.389194 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.389326 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.389977 (+ 651us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.390124 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:49:22.391436 2460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.377740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:45195 (local address 127.0.0.1:44024)
0808 10:49:22.378288 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.378331 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:22.379335 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:22.379864 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:22.379874 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:22.380247 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:22.380847 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.380862 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.381672 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.381675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.382760 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.382771 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.383289 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.383293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.383478 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.384088 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:22.384105 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:22.385873 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:22.388047 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.388053 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.388070 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.388366 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.388693 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.388697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.388699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.388866 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.389327 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:22.389336 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:22.389688 (+ 352us) client_negotiation.cc:769] Sending connection context
0808 10:49:22.389913 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 10:49:22.390269 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":137,"threads_started":1}
I20260808 10:49:22.393005 2456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.188.keytab.
Entry for principal kudu/127.31.237.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.188.keytab.
I20260808 10:49:22.397710 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.188:42365
--webserver_interface=127.31.237.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.188.keytab
--principal=kudu/127.31.237.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:40651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:45195,127.31.237.189:42489,127.31.237.188:42365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.189@KRBTEST.COM for kudu/127.31.237.189@KRBTEST.COM
I20260808 10:49:22.403599 2460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.393543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:42489 (local address 127.0.0.1:56604)
0808 10:49:22.393674 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.393689 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:22.393784 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:22.394531 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:22.394534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:22.394809 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:22.394910 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.394917 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.396244 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.396247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.396811 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.396820 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.397432 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.397436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.397634 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.398136 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:22.398148 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:22.399688 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:22.401943 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.401945 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.401947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.402138 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.402476 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.402478 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.402479 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.402555 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.403191 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:22.403193 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:22.403278 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 10:49:22.403358 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 10:49:22.403472 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":65}
I20260808 10:49:22.403959 2463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.393722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56604 (local address 127.31.237.189:42489)
0808 10:49:22.394075 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.394131 (+ 56us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.394164 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:49:22.394204 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.394208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.394385 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.394506 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.395057 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.396087 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.396972 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.397314 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.397552 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.399835 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.399847 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.399864 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.399895 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.401825 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.402254 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.402264 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.402269 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.402369 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.402681 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.402688 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.402693 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.403067 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.403221 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.403525 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.403703 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":241,"thread_start_us":151,"threads_started":1}
I20260808 10:49:22.406142 2456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:49:22.407066 2460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.406731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42365 (local address 127.0.0.1:53736)
0808 10:49:22.406876 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.406976 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 10:49:22.407291 2379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
I20260808 10:49:22.408200 2460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.407926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42365 (local address 127.0.0.1:53744)
0808 10:49:22.408060 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.408122 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 10:49:22.408360 2456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:42365: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
I20260808 10:49:22.457655 2456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } attempt: 1
I20260808 10:49:22.458642 2460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.458247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42365 (local address 127.0.0.1:53756)
0808 10:49:22.458421 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.458521 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 10:49:22.459465 2460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.459222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42365 (local address 127.0.0.1:53760)
0808 10:49:22.459340 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.459392 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 10:49:22.459628 2456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:42365: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
I20260808 10:49:22.500903 2345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } attempt: 4
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for kudu/127.31.237.189@KRBTEST.COM
I20260808 10:49:22.510370 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.501560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:42489 (local address 127.0.0.1:56610)
0808 10:49:22.501779 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.501796 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:22.501962 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:22.502275 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:22.502278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:22.502486 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:22.502600 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.502656 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.503857 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.503863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.504409 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.504418 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.504978 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.504982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.505102 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.505677 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:22.505690 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:22.507022 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:22.508954 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.508958 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.508961 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.509160 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.509472 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.509475 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.509476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.509555 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.509955 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:22.509958 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:22.510048 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 10:49:22.510130 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:49:22.510238 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260808 10:49:22.510514 2463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.501661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56610 (local address 127.31.237.189:42489)
0808 10:49:22.501796 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.501801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.501886 (+ 85us) server_negotiation.cc:407] Connection header received
0808 10:49:22.502084 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.502092 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.502145 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.502243 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.502804 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.503730 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.504546 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.504836 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.504951 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.507169 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.507174 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.507177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.507207 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.508829 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.509258 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.509262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.509264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.509356 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.509644 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.509647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.509649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.509861 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.509964 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.510214 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.510344 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:49:22.511348 2345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:49:22.512108 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.511813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42365 (local address 127.0.0.1:53766)
0808 10:49:22.511943 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.511995 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 10:49:22.513108 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.512903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42365 (local address 127.0.0.1:53772)
0808 10:49:22.513005 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.513058 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 10:49:22.513258 2345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:42365: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
W20260808 10:49:22.519205 2464 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:49:22.519526 2464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:22.519588 2464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:22.519666 2464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:22.523671 2464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:22.523743 2464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:22.523784 2464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:22.523841 2464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:22.523880 2464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:22.527791 2456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } attempt: 2
I20260808 10:49:22.528540 2460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.528240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42365 (local address 127.0.0.1:53786)
0808 10:49:22.528388 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.528456 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 10:49:22.529250 2460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.529034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42365 (local address 127.0.0.1:53802)
0808 10:49:22.529143 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.529191 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 10:49:22.529397 2456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:42365: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
I20260808 10:49:22.529706 2464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:40651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:45195,127.31.237.189:42489,127.31.237.188:42365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.188.keytab
--principal=kudu/127.31.237.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.188:42365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.237.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:22.530777 2464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:22.531773 2464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:22.543857 2474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:22.543874 2481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:22.544024 2473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:22.544447 2464 server_base.cc:1034] running on GCE node
I20260808 10:49:22.545238 2464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:22.546689 2464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:22.547873 2464 hybrid_clock.cc:648] HybridClock initialized: now 1786186162547841 us; error 48 us; skew 500 ppm
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:22.551985 2464 init.cc:377] Logged in from keytab as kudu/127.31.237.188@KRBTEST.COM (short username kudu)
I20260808 10:49:22.554831 2464 webserver.cc:460] Webserver started at http://127.31.237.188:36375/ using document root <none> and password file <none>
I20260808 10:49:22.555676 2464 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:22.555779 2464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:22.556087 2464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:22.558713 2464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/data/instance:
uuid: "f961151020154f30b5dd96b44a593b59"
format_stamp: "Formatted at 2026-08-08 10:49:22 on dist-test-slave-twwt"
server_key: "478e9049412dbe0b4c10fdfc64c56272"
server_key_iv: "e0abf96ea853accc38fb0d8440454633"
server_key_version: "encryptionkey@0"
I20260808 10:49:22.559545 2464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/wal/instance:
uuid: "f961151020154f30b5dd96b44a593b59"
format_stamp: "Formatted at 2026-08-08 10:49:22 on dist-test-slave-twwt"
server_key: "478e9049412dbe0b4c10fdfc64c56272"
server_key_iv: "e0abf96ea853accc38fb0d8440454633"
server_key_version: "encryptionkey@0"
I20260808 10:49:22.567893 2464 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.005s
I20260808 10:49:22.571679 2494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:22.572402 2345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } attempt: 1
I20260808 10:49:22.573335 2464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 10:49:22.573374 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.573000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42365 (local address 127.0.0.1:53804)
0808 10:49:22.573214 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.573274 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260808 10:49:22.573477 2464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/wal
uuid: "f961151020154f30b5dd96b44a593b59"
format_stamp: "Formatted at 2026-08-08 10:49:22 on dist-test-slave-twwt"
server_key: "478e9049412dbe0b4c10fdfc64c56272"
server_key_iv: "e0abf96ea853accc38fb0d8440454633"
server_key_version: "encryptionkey@0"
I20260808 10:49:22.573612 2464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:22.574479 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.574199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42365 (local address 127.0.0.1:53818)
0808 10:49:22.574341 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.574392 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260808 10:49:22.574703 2345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:42365: Network error: Client connection negotiation failed: client connection to 127.31.237.188:42365: connect: Connection refused (error 111)
I20260808 10:49:22.597952 2464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:22.601585 2464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:22.601815 2464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:22.609531 2464 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.188:42365
I20260808 10:49:22.609826 2572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.188:42365 every 8 connection(s)
I20260808 10:49:22.610641 2464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/data/info.pb
I20260808 10:49:22.613914 2573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:22.617784 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 2464
I20260808 10:49:22.617949 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786185862616645-32694-0/minicluster-data/master-2/wal/instance
I20260808 10:49:22.618326 32694 external_mini_cluster.cc:1428] Setting key 6da4ba636b079421663ad7d64eef4858
I20260808 10:49:22.618575 2573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:49:22.620577 2456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } attempt: 3
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.189@KRBTEST.COM for kudu/127.31.237.188@KRBTEST.COM
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.188@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.188@KRBTEST.COM
I20260808 10:49:22.632997 2460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.621247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42365 (local address 127.0.0.1:53832)
0808 10:49:22.621553 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.621570 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:22.621654 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:22.622054 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:22.622058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:22.622268 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:22.622376 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.622383 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.624305 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.624309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.625200 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.625213 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.625819 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.625823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.625912 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.626682 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:22.626704 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:22.628139 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:22.630781 (+ 2642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.630785 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.630787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.631052 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.631459 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.631462 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.631464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.631561 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.632412 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:22.632415 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:22.632528 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 10:49:22.632636 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 10:49:22.632790 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212}
I20260808 10:49:22.633587 2349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.620862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44040 (local address 127.31.237.190:45195)
0808 10:49:22.620999 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.621006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.621485 (+ 479us) server_negotiation.cc:407] Connection header received
0808 10:49:22.621920 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.621924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.621991 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.622078 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.623307 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.623866 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.625438 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.625722 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.625845 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.629501 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.629504 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.629506 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.629533 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.631301 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.631942 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.631945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.631947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.632040 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.632339 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.632342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.632344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.632553 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.632657 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.633371 (+ 714us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.633478 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:49:22.634474 2578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.620728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:45195 (local address 127.0.0.1:44040)
0808 10:49:22.621317 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.621380 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:22.621682 (+ 302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:22.622297 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:22.622305 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:22.622842 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:22.623142 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.623156 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.623993 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.623999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.625251 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.625265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.626039 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.626046 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.626212 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.626865 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:22.626890 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:22.629158 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:22.631412 (+ 2254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.631423 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.631431 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.631834 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.632136 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.632140 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.632142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.632235 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.632682 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:22.632692 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:22.632897 (+ 205us) client_negotiation.cc:769] Sending connection context
0808 10:49:22.633103 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 10:49:22.633478 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":67,"thread_start_us":87,"threads_started":1}
I20260808 10:49:22.634850 2577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.619833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53828 (local address 127.31.237.188:42365)
0808 10:49:22.620258 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.620283 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.620319 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:49:22.621251 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.621275 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.621696 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.622004 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.623101 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.624169 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.625062 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.625604 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.625786 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.629382 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.629386 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.629414 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.629451 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.632296 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.632861 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.632864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.632866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.632965 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.633777 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.633781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.633782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.633951 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.634162 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.634415 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.634558 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":276,"spinlock_wait_cycles":896,"thread_start_us":139,"threads_started":1}
I20260808 10:49:22.635124 2580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.621517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53832 (local address 127.31.237.188:42365)
0808 10:49:22.621802 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.621810 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.621827 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:49:22.621864 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.621870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.621912 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.622508 (+ 596us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.622663 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.624169 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.625337 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.625687 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.626211 (+ 524us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.628314 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.628321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.628343 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.628368 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.630611 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.631198 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.631206 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.631214 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.631375 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.631690 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.631697 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.631700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.632315 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.632456 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.634760 (+ 2304us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.634913 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":294,"server-negotiator.queue_time_us":230,"spinlock_wait_cycles":3968,"thread_start_us":132,"threads_started":1}
I20260808 10:49:22.636925 2573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:49:22.640166 2456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe: Bootstrap starting.
I20260808 10:49:22.641872 2345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } attempt: 2
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:49:22.643659 2456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:22.644698 2456 log.cc:824] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:22.647213 2456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe: No bootstrap required, opened a new log
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for kudu/127.31.237.188@KRBTEST.COM
I20260808 10:49:22.648061 2349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.637571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44050 (local address 127.31.237.190:45195)
0808 10:49:22.637690 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.637694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.637716 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:49:22.637854 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.637857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.637893 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.637960 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.638736 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.639347 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.640423 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.640627 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.640711 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.643591 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.643597 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.643600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.643627 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.645406 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.645928 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.645931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.645933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.646036 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.646893 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.646896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.646898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.647151 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.647362 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.647757 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.647897 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.188@KRBTEST.COM for kudu/127.31.237.189@KRBTEST.COM
I20260808 10:49:22.651865 2456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } }
I20260808 10:49:22.652107 2456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50621ca72fc24af3a3a1495756d5adbe, State: Initialized, Role: FOLLOWER
I20260808 10:49:22.652473 2456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } }
I20260808 10:49:22.652896 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.642285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42365 (local address 127.0.0.1:53840)
0808 10:49:22.642409 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.642427 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:22.642519 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:22.643046 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:22.643049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:22.643237 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:22.643352 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.643359 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.644415 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.644418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.645220 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.645232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.645806 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.645809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.645885 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.646335 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:22.646354 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:22.648435 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:22.650736 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.650740 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.650742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.651031 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.651547 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.651553 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.651558 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.651651 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.652255 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:22.652258 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:22.652354 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 10:49:22.652440 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:49:22.652569 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 10:49:22.653046 2463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.642214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56618 (local address 127.31.237.189:42489)
0808 10:49:22.642349 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.642354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.642367 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:49:22.642404 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.642408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.642454 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.642561 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.643147 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.644432 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.645521 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.645867 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.645936 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.649272 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.649287 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.649293 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.649323 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.651138 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.651635 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.651639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.651641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.651725 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.652020 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.652023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.652026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.652295 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.652370 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.652754 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.652870 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:49:22.653559 2578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.638932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:42489 (local address 127.0.0.1:56618)
0808 10:49:22.639498 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.639509 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:22.639563 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:22.642579 (+ 3016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:22.642582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:22.642821 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:22.642941 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.642951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.644557 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.644560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.645409 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.645415 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.646095 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.646098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.646213 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.646640 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:22.646655 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:22.649103 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:22.651261 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.651264 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.651266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.651525 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.651838 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.651841 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.651843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.651912 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.652451 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:22.652454 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:22.652542 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 10:49:22.653221 (+ 679us) client_negotiation.cc:240] Negotiation successful
0808 10:49:22.653385 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":3}
I20260808 10:49:22.653967 2456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:22.654117 2577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.642723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53840 (local address 127.31.237.188:42365)
0808 10:49:22.642829 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.642833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.642844 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:49:22.642875 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.642878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.642913 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.643464 (+ 551us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.643545 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.644274 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.645379 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.645688 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.646093 (+ 405us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.648580 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.648583 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.648586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.648617 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.650559 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.651355 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.651358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.651359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.651432 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.651771 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.651774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.651792 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.652011 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.653501 (+ 1490us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.653576 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.653916 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":46}
I20260808 10:49:22.654382 2573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.189@KRBTEST.COM
I20260808 10:49:22.657630 2345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f: Bootstrap starting.
I20260808 10:49:22.660374 2345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:22.661325 2345 log.cc:824] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:22.656864 2589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } } }
I20260808 10:49:22.662264 2589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:49:22.663573 2345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f: No bootstrap required, opened a new log
Aug 08 10:49:22 dist-test-slave-twwt krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186162, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.188@KRBTEST.COM for kudu/127.31.237.188@KRBTEST.COM
I20260808 10:49:22.663470 2352 master_runner.cc:186] Error getting master registration for 127.31.237.190:45195: OK, instance_id { permanent_uuid: "1c0da80e08914545b09a10390cde764f" instance_seqno: 1786186162107630 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:49:22.667246 2352 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 10:49:22.659672 2610 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 10:49:22.661495 2586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.649749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56630 (local address 127.31.237.189:42489)
0808 10:49:22.650319 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.650327 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.650340 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:49:22.650373 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.650377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.650428 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.651333 (+ 905us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.651459 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.652213 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.653281 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.653589 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.656938 (+ 3349us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.657066 (+ 128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.657069 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.657072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.657095 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.659423 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.659933 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.659936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.659938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.660039 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.660397 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.660401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.660402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.660626 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.660831 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.661046 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.661174 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":122,"threads_started":1}
W20260808 10:49:22.667337 2610 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:49:22.668816 2345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } }
I20260808 10:49:22.669086 2345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c0da80e08914545b09a10390cde764f, State: Initialized, Role: FOLLOWER
I20260808 10:49:22.669452 2345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } }
I20260808 10:49:22.671222 2578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.655082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:42365 (local address 127.0.0.1:53854)
0808 10:49:22.655279 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:22.655292 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:22.655364 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:22.662196 (+ 6832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:22.662199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:22.662382 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:22.662476 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.662485 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.663503 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.663506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.664276 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:22.664288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.664971 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.664974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:22.665046 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.665508 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:22.665525 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:22.667131 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:22.669427 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.669433 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.669436 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.669667 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.670111 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:22.670115 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:22.670117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:22.670204 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:22.670716 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:22.670718 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:22.670817 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 10:49:22.670923 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 10:49:22.671058 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260808 10:49:22.671536 2345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:22.671679 2577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.661800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53854 (local address 127.31.237.188:42365)
0808 10:49:22.661925 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.661931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.661947 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:49:22.661987 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.661989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.662031 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.662105 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.662713 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.663389 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.664450 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.664766 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.664870 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.667613 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.667615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.667617 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.667638 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.669319 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.669916 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.669922 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.669924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.670013 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.670303 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.670306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.670308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.670542 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.670678 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.671422 (+ 744us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.671551 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:49:22.672523 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:22.674887 2309 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1c0da80e08914545b09a10390cde764f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44050
I20260808 10:49:22.675006 2573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59: Bootstrap starting.
I20260808 10:49:22.675187 2309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 10:49:22.675298 2309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:22.676600 2309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } }
W20260808 10:49:22.677058 2633 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:49:22.677126 2633 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:49:22.677377 2614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } } }
I20260808 10:49:22.677484 2614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:49:22.677587 2309 leader_election.cc:288] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [CANDIDATE]: Term 1 election: Requested vote from peers 50621ca72fc24af3a3a1495756d5adbe (127.31.237.189:42489), f961151020154f30b5dd96b44a593b59 (127.31.237.188:42365)
I20260808 10:49:22.678030 2573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:22.678552 2416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c0da80e08914545b09a10390cde764f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "50621ca72fc24af3a3a1495756d5adbe"
I20260808 10:49:22.678821 2416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:22.679185 2573 log.cc:824] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:22.680100 2416 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c0da80e08914545b09a10390cde764f in term 1.
I20260808 10:49:22.680477 2268 leader_election.cc:302] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c0da80e08914545b09a10390cde764f, 50621ca72fc24af3a3a1495756d5adbe; no voters:
I20260808 10:49:22.680712 2614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:22.680925 2614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [term 1 LEADER]: Becoming Leader. State: Replica: 1c0da80e08914545b09a10390cde764f, State: Running, Role: LEADER
I20260808 10:49:22.681259 2614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } }
I20260808 10:49:22.683149 2614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c0da80e08914545b09a10390cde764f. Latest consensus state: current_term: 1 leader_uuid: "1c0da80e08914545b09a10390cde764f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } } }
I20260808 10:49:22.683262 2614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:22.688684 2649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:22.689671 2649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:22.693133 2573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59: No bootstrap required, opened a new log
I20260808 10:49:22.688839 2233 master_runner.cc:186] Error getting master registration for 127.31.237.188:42365: OK, instance_id { permanent_uuid: "f961151020154f30b5dd96b44a593b59" instance_seqno: 1786186162607346 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:49:22.693460 2233 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:49:22.697695 2573 raft_consensus.cc:348] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } }
I20260808 10:49:22.698052 2573 raft_consensus.cc:729] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f961151020154f30b5dd96b44a593b59, State: Initialized, Role: FOLLOWER
I20260808 10:49:22.698467 2573 consensus_queue.cc:260] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } }
I20260808 10:49:22.700129 2416 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe [term 1 FOLLOWER]: Refusing update from remote peer 1c0da80e08914545b09a10390cde764f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:49:22.700989 2636 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [LEADER]: Connected to new peer: Peer: permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:49:22.706243 2589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c0da80e08914545b09a10390cde764f. Latest consensus state: current_term: 1 leader_uuid: "1c0da80e08914545b09a10390cde764f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } } }
I20260808 10:49:22.706408 2589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:49:22.707530 2573 sys_catalog.cc:564] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:22.707585 2532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1c0da80e08914545b09a10390cde764f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f961151020154f30b5dd96b44a593b59"
I20260808 10:49:22.707824 2532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:22.709463 2532 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c0da80e08914545b09a10390cde764f in term 1.
I20260808 10:49:22.710557 2531 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59 [term 1 FOLLOWER]: Refusing update from remote peer 1c0da80e08914545b09a10390cde764f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:49:22.712080 2577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.686880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53856 (local address 127.31.237.188:42365)
0808 10:49:22.687043 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.687048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.694930 (+ 7882us) server_negotiation.cc:407] Connection header received
0808 10:49:22.695160 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.695164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.695215 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.695268 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.696351 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.697142 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.698370 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.698758 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.698860 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.703955 (+ 5095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.703959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.703962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.704000 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.706224 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.706744 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.706748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.706750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.706931 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.710680 (+ 3749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.710685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.710687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.710964 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.711084 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.711801 (+ 717us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.711928 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 10:49:22.712280 2636 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:49:22.714176 2349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.687644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44054 (local address 127.31.237.190:45195)
0808 10:49:22.687944 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.687950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.698867 (+ 10917us) server_negotiation.cc:407] Connection header received
0808 10:49:22.699069 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.699074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.699128 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.699180 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.700181 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.700874 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.702515 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.702863 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.702964 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.707993 (+ 5029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.707998 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.708001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.708030 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.709734 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.711726 (+ 1992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.711730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.711732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.711826 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.712429 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.712433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.712435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.712684 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.712840 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.713850 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.714032 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230}
I20260808 10:49:22.720221 2586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:22.686155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56640 (local address 127.31.237.189:42489)
0808 10:49:22.686285 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.686301 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.700340 (+ 14039us) server_negotiation.cc:407] Connection header received
0808 10:49:22.700507 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.700511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.700562 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.700612 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.701549 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.702292 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.706750 (+ 4458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:22.707116 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:22.707194 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:22.714145 (+ 6951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:22.714149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:22.714151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:22.714182 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:22.715931 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.718763 (+ 2832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.718770 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.718773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.718884 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:22.719209 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:22.719213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:22.719214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:22.719442 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:22.719576 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:22.719946 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 10:49:22.720078 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 10:49:22.720605 2676 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:49:22.720691 2676 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:49:22.721580 2656 sys_catalog.cc:455] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c0da80e08914545b09a10390cde764f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } } }
I20260808 10:49:22.721707 2656 sys_catalog.cc:458] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:49:22.721794 2656 sys_catalog.cc:455] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c0da80e08914545b09a10390cde764f. Latest consensus state: current_term: 1 leader_uuid: "1c0da80e08914545b09a10390cde764f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } } }
I20260808 10:49:22.721863 2656 sys_catalog.cc:458] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:49:22.723129 2614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c0da80e08914545b09a10390cde764f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } } }
I20260808 10:49:22.723275 2614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:22.723392 2614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c0da80e08914545b09a10390cde764f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } } }
I20260808 10:49:22.723477 2614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:22.723452 2589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c0da80e08914545b09a10390cde764f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } } }
I20260808 10:49:22.723554 2589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:49:22.723955 2655 mvcc.cc:205] Tried to move back new op lower bound from 7316218522407088128 to 7316218522345938944. Current Snapshot: MvccSnapshot[applied={T|T < 7316218522407088128}]
I20260808 10:49:22.724584 2656 sys_catalog.cc:455] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c0da80e08914545b09a10390cde764f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c0da80e08914545b09a10390cde764f" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 45195 } } peers { permanent_uuid: "50621ca72fc24af3a3a1495756d5adbe" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 42489 } } peers { permanent_uuid: "f961151020154f30b5dd96b44a593b59" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 42365 } } }
I20260808 10:49:22.724690 2656 sys_catalog.cc:458] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:49:22.724195 2659 mvcc.cc:205] Tried to move back new op lower bound from 7316218522407088128 to 7316218522345938944. Current Snapshot: MvccSnapshot[applied={T|T < 7316218522407088128}]
I20260808 10:49:22.728458 2649 catalog_manager.cc:1347] Generated new cluster ID: cc2abbc61e78455fbb087b21145b08fa
I20260808 10:49:22.728523 2649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:22.746672 2649 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:22.748265 2649 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:22.763230 2649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c0da80e08914545b09a10390cde764f: Generated new TSK 0
I20260808 10:49:22.764256 2649 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:49:22.806910 2577 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:22.803370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53862 (local address 127.31.237.188:42365)
0808 10:49:22.803510 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.803516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.805624 (+ 2108us) server_negotiation.cc:407] Connection header received
0808 10:49:22.805868 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.805872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.805929 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.805990 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.806754 (+ 764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53862: BlockingRecv error: recv got EOF from 127.0.0.1:53862 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 10:49:22.809171 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:22.804196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56650 (local address 127.31.237.189:42489)
0808 10:49:22.804325 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.804330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.807693 (+ 3363us) server_negotiation.cc:407] Connection header received
0808 10:49:22.807961 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.807965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.808031 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.808087 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.809040 (+ 953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56650: BlockingRecv error: recv got EOF from 127.0.0.1:56650 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 10:49:22.811009 2349 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:22.804929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44058 (local address 127.31.237.190:45195)
0808 10:49:22.805053 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.805058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.809828 (+ 4770us) server_negotiation.cc:407] Connection header received
0808 10:49:22.810099 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.810104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.810160 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.810215 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.810877 (+ 662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44058: BlockingRecv error: recv got EOF from 127.0.0.1:44058 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 10:49:22.816272 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:22.814456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56654 (local address 127.31.237.189:42489)
0808 10:49:22.814572 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.814577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.815197 (+ 620us) server_negotiation.cc:407] Connection header received
0808 10:49:22.815439 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.815443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.815524 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.815581 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.816157 (+ 576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56654: BlockingRecv error: recv got EOF from 127.0.0.1:56654 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 10:49:22.821945 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:22.820204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56666 (local address 127.31.237.189:42489)
0808 10:49:22.820315 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.820319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.820937 (+ 618us) server_negotiation.cc:407] Connection header received
0808 10:49:22.821138 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.821142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.821198 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.821255 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.821825 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56666: BlockingRecv error: recv got EOF from 127.0.0.1:56666 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 10:49:22.827986 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:22.826197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56670 (local address 127.31.237.189:42489)
0808 10:49:22.826313 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.826318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.827001 (+ 683us) server_negotiation.cc:407] Connection header received
0808 10:49:22.827202 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.827205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.827265 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.827320 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.827869 (+ 549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56670: BlockingRecv error: recv got EOF from 127.0.0.1:56670 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 10:49:22.836094 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:22.834265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56676 (local address 127.31.237.189:42489)
0808 10:49:22.834417 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.834421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.835061 (+ 640us) server_negotiation.cc:407] Connection header received
0808 10:49:22.835285 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.835289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.835347 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.835403 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.835979 (+ 576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56676: BlockingRecv error: recv got EOF from 127.0.0.1:56676 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260808 10:49:22.848502 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:22.846636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56686 (local address 127.31.237.189:42489)
0808 10:49:22.846780 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.846785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.847410 (+ 625us) server_negotiation.cc:407] Connection header received
0808 10:49:22.847633 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.847638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.847696 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.847772 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.848369 (+ 597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56686: BlockingRecv error: recv got EOF from 127.0.0.1:56686 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 10:49:22.869534 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:22.867442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56690 (local address 127.31.237.189:42489)
0808 10:49:22.867599 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.867604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.868316 (+ 712us) server_negotiation.cc:407] Connection header received
0808 10:49:22.868598 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.868602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.868668 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.868736 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.869392 (+ 656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56690: BlockingRecv error: recv got EOF from 127.0.0.1:56690 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 10:49:22.905478 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:22.904129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56704 (local address 127.31.237.189:42489)
0808 10:49:22.904277 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.904282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.904550 (+ 268us) server_negotiation.cc:407] Connection header received
0808 10:49:22.904674 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.904678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.904760 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.904855 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.905346 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56704: BlockingRecv error: recv got EOF from 127.0.0.1:56704 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 10:49:22.973390 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:22.971839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56712 (local address 127.31.237.189:42489)
0808 10:49:22.972038 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:22.972045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:22.972177 (+ 132us) server_negotiation.cc:407] Connection header received
0808 10:49:22.972402 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:22.972409 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:22.972479 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:22.972582 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:22.973255 (+ 673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56712: BlockingRecv error: recv got EOF from 127.0.0.1:56712 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 10:49:23.105346 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:23.104163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56718 (local address 127.31.237.189:42489)
0808 10:49:23.104400 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:23.104407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:23.104427 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:49:23.104493 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:23.104496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:23.104545 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:23.104647 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:23.105211 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56718: BlockingRecv error: recv got EOF from 127.0.0.1:56718 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 10:49:23.365720 2586 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:23.363946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56724 (local address 127.31.237.189:42489)
0808 10:49:23.364096 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:23.364101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:23.364662 (+ 561us) server_negotiation.cc:407] Connection header received
0808 10:49:23.364788 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:23.364792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:23.364870 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:23.364992 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:23.365598 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56724: BlockingRecv error: recv got EOF from 127.0.0.1:56724 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:49:23.668872 2610 catalog_manager.cc:1259] Loaded cluster ID: cc2abbc61e78455fbb087b21145b08fa
I20260808 10:49:23.668973 2610 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe: loading cluster ID for follower catalog manager: success
I20260808 10:49:23.670997 2610 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe: acquiring CA information for follower catalog manager: success
I20260808 10:49:23.672457 2610 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 50621ca72fc24af3a3a1495756d5adbe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:49:23.722124 2676 catalog_manager.cc:1259] Loaded cluster ID: cc2abbc61e78455fbb087b21145b08fa
I20260808 10:49:23.722251 2676 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59: loading cluster ID for follower catalog manager: success
I20260808 10:49:23.723997 2676 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59: acquiring CA information for follower catalog manager: success
I20260808 10:49:23.725207 2676 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f961151020154f30b5dd96b44a593b59: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:49:23.883976 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:23.880105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56740 (local address 127.31.237.189:42489)
0808 10:49:23.880376 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:23.880379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:23.880398 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:49:23.880550 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:23.880553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:23.880689 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:23.880791 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:49:23.881050 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:23.881574 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:23.882443 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:23.882697 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:23.882774 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:23.883075 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:49:23.883539 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:49:23.883632 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:23.883718 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 10:49:23.883769 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":77,"threads_started":1}
I20260808 10:49:23.890908 2798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:23.887290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53874 (local address 127.31.237.188:42365)
0808 10:49:23.887520 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:23.887525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:23.887540 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:49:23.887620 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:23.887622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:23.887732 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:23.887832 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:49:23.888036 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:23.888538 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:23.889490 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:23.889717 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:23.889799 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:23.890075 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:49:23.890492 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:49:23.890585 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:23.890739 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 10:49:23.890791 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260808 10:49:23.898038 2798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:23.893928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53888 (local address 127.31.237.188:42365)
0808 10:49:23.894052 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:23.894057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:23.894495 (+ 438us) server_negotiation.cc:407] Connection header received
0808 10:49:23.894664 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:23.894667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:23.894707 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:23.894797 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:49:23.895044 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:23.895728 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:23.896608 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:23.896934 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:23.897038 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:23.897546 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:49:23.897620 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:49:23.897663 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:23.897823 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 10:49:23.897875 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:49:23.898653 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:23.893823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56746 (local address 127.31.237.189:42489)
0808 10:49:23.894106 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:23.894110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:23.894813 (+ 703us) server_negotiation.cc:407] Connection header received
0808 10:49:23.894930 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:23.894933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:23.895012 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:23.895089 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:49:23.895376 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:23.896179 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:23.897474 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:23.897706 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:23.897816 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:23.898152 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:49:23.898240 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:49:23.898290 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:23.898459 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 10:49:23.898515 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260808 10:49:23.899323 2809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:23.893824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44072 (local address 127.31.237.190:45195)
0808 10:49:23.894281 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:23.894286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:23.894531 (+ 245us) server_negotiation.cc:407] Connection header received
0808 10:49:23.894961 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:23.894967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:23.895155 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:23.895283 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:49:23.895533 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:23.896298 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:23.897528 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:23.897908 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:23.898011 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:23.898371 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:49:23.898736 (+ 365us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:49:23.898828 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:23.899091 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 10:49:23.899180 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":71,"threads_started":1}
I20260808 10:49:23.902383 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 2233
I20260808 10:49:23.914088 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 2352
I20260808 10:49:23.929457 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 2464
2026-08-08T10:49:23Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3448 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:49:23 dist-test-slave-twwt krb5kdc[2816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:49:23 dist-test-slave-twwt krb5kdc[2816](info): set up 2 sockets
Aug 08 10:49:23 dist-test-slave-twwt krb5kdc[2816](info): commencing operation
krb5kdc: starting...
W20260808 10:49:25.187776 32694 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/test-user.keytab.
Aug 08 10:49:25 dist-test-slave-twwt krb5kdc[2816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:49:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:49:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.237.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
Entry for principal kudu/127.31.237.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab.
I20260808 10:49:25.326102 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:32959
--webserver_interface=127.31.237.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:35079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:32959,127.31.237.189:36265,127.31.237.188:36823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:25.447307 2837 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:49:25.447654 2837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:25.447741 2837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:25.447821 2837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:25.451921 2837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:25.452008 2837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:25.452047 2837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:25.452086 2837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:25.452117 2837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:25.457538 2837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:35079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:32959,127.31.237.189:36265,127.31.237.188:36823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.190.keytab
--principal=kudu/127.31.237.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.190:32959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.237.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:25.458810 2837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:25.459928 2837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:25.469023 2847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:25.468986 2846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:25.469147 2837 server_base.cc:1034] running on GCE node
W20260808 10:49:25.469064 2852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:25.469864 2837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:25.471083 2837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:25.472250 2837 hybrid_clock.cc:648] HybridClock initialized: now 1786186165472219 us; error 48 us; skew 500 ppm
Aug 08 10:49:25 dist-test-slave-twwt krb5kdc[2816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:25.479519 2837 init.cc:377] Logged in from keytab as kudu/127.31.237.190@KRBTEST.COM (short username kudu)
I20260808 10:49:25.480481 2837 webserver.cc:460] Webserver started at http://127.31.237.190:36573/ using document root <none> and password file <none>
I20260808 10:49:25.481086 2837 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:25.481148 2837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:25.481397 2837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:25.483646 2837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/data/instance:
uuid: "2bf7a7bb007940c5ae5c0035e00086dd"
format_stamp: "Formatted at 2026-08-08 10:49:25 on dist-test-slave-twwt"
server_key: "10e45206bd1eeaf466339523e2ae833c"
server_key_iv: "2bbc7029cc298bd64b7617c5ca2b55ab"
server_key_version: "encryptionkey@0"
I20260808 10:49:25.484242 2837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/wal/instance:
uuid: "2bf7a7bb007940c5ae5c0035e00086dd"
format_stamp: "Formatted at 2026-08-08 10:49:25 on dist-test-slave-twwt"
server_key: "10e45206bd1eeaf466339523e2ae833c"
server_key_iv: "2bbc7029cc298bd64b7617c5ca2b55ab"
server_key_version: "encryptionkey@0"
I20260808 10:49:25.488204 2837 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 10:49:25.490942 2862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:25.492285 2837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 10:49:25.492436 2837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/wal
uuid: "2bf7a7bb007940c5ae5c0035e00086dd"
format_stamp: "Formatted at 2026-08-08 10:49:25 on dist-test-slave-twwt"
server_key: "10e45206bd1eeaf466339523e2ae833c"
server_key_iv: "2bbc7029cc298bd64b7617c5ca2b55ab"
server_key_version: "encryptionkey@0"
I20260808 10:49:25.492564 2837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:25.508169 2837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:25.511373 2837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:25.511583 2837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:25.519375 2837 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.190:32959
I20260808 10:49:25.519690 2950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.190:32959 every 8 connection(s)
I20260808 10:49:25.520447 2837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/data/info.pb
I20260808 10:49:25.523586 2951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:25.523882 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 2837
I20260808 10:49:25.524009 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-0/wal/instance
I20260808 10:49:25.524335 32694 external_mini_cluster.cc:1428] Setting key 3ace782c9734c0de4c19bf09c884a916
I20260808 10:49:25.528468 2951 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.237.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.189@KRBTEST.COM" created.
Aug 08 10:49:25 dist-test-slave-twwt krb5kdc[2816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:49:25.544415 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.530971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37628 (local address 127.31.237.190:32959)
0808 10:49:25.531425 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:25.531445 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:25.531744 (+ 299us) server_negotiation.cc:407] Connection header received
0808 10:49:25.532829 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:25.532833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:25.533027 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:25.533223 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:25.534144 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.535162 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.536136 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.536633 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.536847 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:25.539583 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:25.539589 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:25.539599 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:25.539620 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:25.541251 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:25.541745 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:25.541751 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:25.541753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:25.541861 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:25.542174 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:25.542179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:25.542181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:25.542554 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:25.542766 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:25.543132 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 10:49:25.543385 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":301,"thread_start_us":156,"threads_started":1}
I20260808 10:49:25.544415 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.530972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:32959 (local address 127.0.0.1:37628)
0808 10:49:25.531584 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.531622 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:25.532496 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:25.533228 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:25.533233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:25.533679 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:25.533989 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:25.534004 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.535296 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.535300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:25.536011 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:25.536020 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.536799 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.536803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:25.537015 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:25.537623 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:25.537651 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:25.539422 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:25.541380 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:25.541392 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:25.541403 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:25.541632 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:25.541962 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:25.541965 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:25.541967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:25.542082 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:25.542734 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:25.542740 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:25.542896 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 10:49:25.543068 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 10:49:25.543436 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":134,"spinlock_wait_cycles":2816,"thread_start_us":180,"threads_started":1}
I20260808 10:49:25.546981 2951 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:49:25.547768 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.547519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:36265 (local address 127.0.0.1:43794)
0808 10:49:25.547637 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.547705 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:36265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 10:49:25.548038 2867 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.237.189:36265: connect: Connection refused (error 111)
I20260808 10:49:25.548852 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.548665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:36265 (local address 127.0.0.1:43798)
0808 10:49:25.548758 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.548800 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:36265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 10:49:25.549124 2951 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.189:36265: Network error: Client connection negotiation failed: client connection to 127.31.237.189:36265: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.189.keytab.
Entry for principal kudu/127.31.237.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.189.keytab.
I20260808 10:49:25.553720 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.189:36265
--webserver_interface=127.31.237.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.189.keytab
--principal=kudu/127.31.237.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:35079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:32959,127.31.237.189:36265,127.31.237.188:36823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:49:25.598368 2951 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } attempt: 1
I20260808 10:49:25.599290 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.598975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:36265 (local address 127.0.0.1:43814)
0808 10:49:25.599139 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.599209 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:36265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 10:49:25.600184 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.599952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:36265 (local address 127.0.0.1:43818)
0808 10:49:25.600077 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.600124 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:36265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 10:49:25.600374 2951 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.189:36265: Network error: Client connection negotiation failed: client connection to 127.31.237.189:36265: connect: Connection refused (error 111)
I20260808 10:49:25.668560 2951 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } attempt: 2
I20260808 10:49:25.669447 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.669135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:36265 (local address 127.0.0.1:43830)
0808 10:49:25.669285 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.669353 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:36265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 10:49:25.670205 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.669993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:36265 (local address 127.0.0.1:43846)
0808 10:49:25.670103 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.670153 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.189:36265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 10:49:25.670398 2951 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.189:36265: Network error: Client connection negotiation failed: client connection to 127.31.237.189:36265: connect: Connection refused (error 111)
W20260808 10:49:25.673732 2965 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:49:25.674079 2965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:49:25.674144 2965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:25.674224 2965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:49:25.678475 2965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:25.678578 2965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:25.678618 2965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:25.678704 2965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:25.678743 2965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:25.684559 2965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:35079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:32959,127.31.237.189:36265,127.31.237.188:36823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.189.keytab
--principal=kudu/127.31.237.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.189:36265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.237.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:25.685786 2965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:25.686931 2965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:25.696451 2973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:25.696451 2972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:25.696456 2978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:25.697065 2965 server_base.cc:1034] running on GCE node
I20260808 10:49:25.697695 2965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:25.699062 2965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:25.700273 2965 hybrid_clock.cc:648] HybridClock initialized: now 1786186165700247 us; error 37 us; skew 500 ppm
Aug 08 10:49:25 dist-test-slave-twwt krb5kdc[2816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:25.703341 2965 init.cc:377] Logged in from keytab as kudu/127.31.237.189@KRBTEST.COM (short username kudu)
I20260808 10:49:25.704289 2965 webserver.cc:460] Webserver started at http://127.31.237.189:41271/ using document root <none> and password file <none>
I20260808 10:49:25.704869 2965 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:25.704931 2965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:25.705116 2965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:25.707464 2965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/data/instance:
uuid: "bbadfbf3016e4d03ba0bea28347ad7fc"
format_stamp: "Formatted at 2026-08-08 10:49:25 on dist-test-slave-twwt"
server_key: "572aef7962f5d601fa4115056a7983fc"
server_key_iv: "db21aae969e870cb8562d0910c9a1174"
server_key_version: "encryptionkey@0"
I20260808 10:49:25.708058 2965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/wal/instance:
uuid: "bbadfbf3016e4d03ba0bea28347ad7fc"
format_stamp: "Formatted at 2026-08-08 10:49:25 on dist-test-slave-twwt"
server_key: "572aef7962f5d601fa4115056a7983fc"
server_key_iv: "db21aae969e870cb8562d0910c9a1174"
server_key_version: "encryptionkey@0"
I20260808 10:49:25.712332 2965 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 10:49:25.715166 2989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:25.716548 2965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 10:49:25.716684 2965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/wal
uuid: "bbadfbf3016e4d03ba0bea28347ad7fc"
format_stamp: "Formatted at 2026-08-08 10:49:25 on dist-test-slave-twwt"
server_key: "572aef7962f5d601fa4115056a7983fc"
server_key_iv: "db21aae969e870cb8562d0910c9a1174"
server_key_version: "encryptionkey@0"
I20260808 10:49:25.716789 2965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:25.732796 2965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:25.735918 2965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:25.736088 2965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:25.743726 2965 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.189:36265
I20260808 10:49:25.744042 3078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.189:36265 every 8 connection(s)
I20260808 10:49:25.744808 2965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/data/info.pb
I20260808 10:49:25.747826 3080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:25.751423 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 2965
I20260808 10:49:25.751526 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-1/wal/instance
I20260808 10:49:25.751807 32694 external_mini_cluster.cc:1428] Setting key 7d00c55348dffc2bd06b3f2f4053a9d6
I20260808 10:49:25.751816 3080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.237.188@KRBTEST.COM; defaulting to no policy
I20260808 10:49:25.761552 2951 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.237.188@KRBTEST.COM" created.
Aug 08 10:49:25 dist-test-slave-twwt krb5kdc[2816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.189@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
Aug 08 10:49:25 dist-test-slave-twwt krb5kdc[2816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for kudu/127.31.237.189@KRBTEST.COM
I20260808 10:49:25.770207 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.753997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37642 (local address 127.31.237.190:32959)
0808 10:49:25.754803 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:25.754811 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:25.755087 (+ 276us) server_negotiation.cc:407] Connection header received
0808 10:49:25.756098 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:25.756105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:25.756181 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:25.756291 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:25.757680 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.758657 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.759827 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.760236 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.760359 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:25.765585 (+ 5226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:25.765592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:25.765594 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:25.765627 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:25.767484 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:25.768196 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:25.768201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:25.768204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:25.768319 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:25.768734 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:25.768741 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:25.768746 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:25.769045 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:25.769158 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:25.769884 (+ 726us) server_negotiation.cc:299] Negotiation successful
0808 10:49:25.770015 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673}
I20260808 10:49:25.771682 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.754353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:32959 (local address 127.0.0.1:37642)
0808 10:49:25.754909 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.754965 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:25.755849 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:25.756523 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:25.756530 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:25.756930 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:25.757505 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:25.757519 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.758798 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.758801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:25.759683 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:25.759692 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.760399 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.760402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:25.760551 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:25.761196 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:25.761214 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:25.765435 (+ 4221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:25.767610 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:25.767620 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:25.767641 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:25.768056 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:25.768444 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:25.768450 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:25.768455 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:25.768622 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:25.769177 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:25.769186 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:25.769559 (+ 373us) client_negotiation.cc:769] Sending connection context
0808 10:49:25.769818 (+ 259us) client_negotiation.cc:240] Negotiation successful
0808 10:49:25.770546 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":142,"threads_started":1}
I20260808 10:49:25.773483 3080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:49:25.774319 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.762076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:36265 (local address 127.0.0.1:43852)
0808 10:49:25.762238 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.762256 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:25.762351 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:25.763194 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:25.763196 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:25.763349 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:25.763466 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:25.763471 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.764709 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.764713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:25.765510 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:25.765526 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.766364 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.766371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:25.766528 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:25.767618 (+ 1090us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:25.767630 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:25.769559 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:25.772385 (+ 2826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:25.772388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:25.772390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:25.772625 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:25.773025 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:25.773032 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:25.773037 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:25.773158 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:25.773733 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:25.773736 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:25.773873 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 10:49:25.773993 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 10:49:25.774143 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 10:49:25.774820 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.762205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43852 (local address 127.31.237.189:36265)
0808 10:49:25.762670 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:25.762682 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:25.762706 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:49:25.762808 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:25.762813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:25.763028 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:25.763182 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:25.763590 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.764576 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.765640 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.766226 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.766351 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:25.769735 (+ 3384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:25.769749 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:25.769769 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:25.769803 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:25.772215 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:25.772760 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:25.772767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:25.772769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:25.772896 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:25.773269 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:25.773273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:25.773274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:25.773576 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:25.773766 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:25.774126 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 10:49:25.774587 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":159,"threads_started":1}
I20260808 10:49:25.776800 2951 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:49:25.777578 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.777293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:39908)
0808 10:49:25.777429 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.777486 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 10:49:25.778452 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.778140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:39918)
0808 10:49:25.778338 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.778388 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260808 10:49:25.778736 2951 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:36823: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
Aug 08 10:49:25 dist-test-slave-twwt krb5kdc[2816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.189@KRBTEST.COM for kudu/127.31.237.189@KRBTEST.COM
I20260808 10:49:25.784937 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.774072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:36265 (local address 127.0.0.1:43856)
0808 10:49:25.774300 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.774312 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:25.774393 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:25.775144 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:25.775146 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:25.775319 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:25.775402 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:25.775408 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.776383 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.776386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:25.777179 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:25.777190 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.777842 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.777846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:25.777946 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:25.778552 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:25.778569 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:25.780622 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:25.783262 (+ 2640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:25.783265 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:25.783267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:25.783478 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:25.783755 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:25.783757 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:25.783758 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:25.783877 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:25.784353 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:25.784356 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:25.784533 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 10:49:25.784636 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 10:49:25.784752 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260808 10:49:25.784951 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.774301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43856 (local address 127.31.237.189:36265)
0808 10:49:25.774943 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:25.774954 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:25.774971 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:49:25.775007 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:25.775010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:25.775050 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:25.775135 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:25.775512 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.776255 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.777306 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.777612 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.777735 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:25.780803 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:25.780807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:25.780809 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:25.780832 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:25.783115 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:25.783570 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:25.783573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:25.783574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:25.783655 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:25.783980 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:25.783983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:25.783984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:25.784201 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:25.784302 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:25.784701 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 10:49:25.784806 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":137,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.237.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.188.keytab.
Entry for principal kudu/127.31.237.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.188.keytab.
I20260808 10:49:25.785813 3080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:49:25.786651 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.786292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:39924)
0808 10:49:25.786429 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.786537 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 10:49:25.786859 32694 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
/tmp/dist-test-taskDI8wVd/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.188:36823
--webserver_interface=127.31.237.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.188.keytab
--principal=kudu/127.31.237.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.237.148:35079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.237.190:32959,127.31.237.189:36265,127.31.237.188:36823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:49:25.786906 2996 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
I20260808 10:49:25.787899 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.787599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:39928)
0808 10:49:25.787748 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.787810 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 10:49:25.788080 3080 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:36823: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
I20260808 10:49:25.809921 2951 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } attempt: 1
I20260808 10:49:25.810909 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.810550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:39930)
0808 10:49:25.810758 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.810822 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 10:49:25.811676 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.811443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:39944)
0808 10:49:25.811562 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.811617 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 10:49:25.811811 2951 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:36823: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
I20260808 10:49:25.837369 3080 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } attempt: 1
I20260808 10:49:25.838325 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.838015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:39956)
0808 10:49:25.838176 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.838242 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 10:49:25.839195 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.838958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:39964)
0808 10:49:25.839087 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.839141 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 10:49:25.839332 3080 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:36823: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
I20260808 10:49:25.887019 2951 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } attempt: 2
I20260808 10:49:25.887924 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.887607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:39970)
0808 10:49:25.887767 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.887841 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 10:49:25.888789 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.888583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:39982)
0808 10:49:25.888690 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.888736 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 10:49:25.888970 2951 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:36823: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
I20260808 10:49:25.907490 3080 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } attempt: 2
W20260808 10:49:25.908003 3095 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:49:25.908411 3095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260808 10:49:25.908413 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.908093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:39990)
0808 10:49:25.908246 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.908308 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 10:49:25.908484 3095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:49:25.908562 3095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 10:49:25.909161 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.908948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:39992)
0808 10:49:25.909057 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.909110 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 10:49:25.909281 3080 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.237.188:36823: Network error: Client connection negotiation failed: client connection to 127.31.237.188:36823: connect: Connection refused (error 111)
W20260808 10:49:25.912842 3095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:49:25.912925 3095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:49:25.912974 3095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:49:25.913012 3095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:49:25.913044 3095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:49:25.918339 3095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.237.148:35079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.237.190:32959,127.31.237.189:36265,127.31.237.188:36823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/krb5kdc/kudu_127.31.237.188.keytab
--principal=kudu/127.31.237.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.237.188:36823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.237.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 09:39:54 UTC on ccycloud.quasar-npkpbv.root.comops.site
build id 4574358
I20260808 10:49:25.919476 3095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:49:25.920627 3095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:49:25.929405 3103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:25.929430 3109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:49:25.929433 3106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:49:25.929715 3095 server_base.cc:1034] running on GCE node
I20260808 10:49:25.930426 3095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:49:25.931595 3095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:49:25.932791 3095 hybrid_clock.cc:648] HybridClock initialized: now 1786186165932774 us; error 29 us; skew 500 ppm
Aug 08 10:49:25 dist-test-slave-twwt krb5kdc[2816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:49:25.935771 3095 init.cc:377] Logged in from keytab as kudu/127.31.237.188@KRBTEST.COM (short username kudu)
I20260808 10:49:25.936692 3095 webserver.cc:460] Webserver started at http://127.31.237.188:33523/ using document root <none> and password file <none>
I20260808 10:49:25.937288 3095 fs_manager.cc:359] Metadata directory not provided
I20260808 10:49:25.937383 3095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:49:25.937635 3095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:49:25.939841 3095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/data/instance:
uuid: "a06d85c752284403aac72e0391a3817e"
format_stamp: "Formatted at 2026-08-08 10:49:25 on dist-test-slave-twwt"
server_key: "6de338542a9a5c21a3cf2c2deb6957b0"
server_key_iv: "41a0abb3ea2990ee7687c7881340bffe"
server_key_version: "encryptionkey@0"
I20260808 10:49:25.940465 3095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/wal/instance:
uuid: "a06d85c752284403aac72e0391a3817e"
format_stamp: "Formatted at 2026-08-08 10:49:25 on dist-test-slave-twwt"
server_key: "6de338542a9a5c21a3cf2c2deb6957b0"
server_key_iv: "41a0abb3ea2990ee7687c7881340bffe"
server_key_version: "encryptionkey@0"
I20260808 10:49:25.944386 3095 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 10:49:25.946913 3118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:49:25.948148 3095 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 10:49:25.948310 3095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/data,/tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/wal
uuid: "a06d85c752284403aac72e0391a3817e"
format_stamp: "Formatted at 2026-08-08 10:49:25 on dist-test-slave-twwt"
server_key: "6de338542a9a5c21a3cf2c2deb6957b0"
server_key_iv: "41a0abb3ea2990ee7687c7881340bffe"
server_key_version: "encryptionkey@0"
I20260808 10:49:25.948432 3095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:49:25.972565 3095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:49:25.976353 3095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:49:25.976603 3095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:49:25.984722 3095 rpc_server.cc:307] RPC server started. Bound to: 127.31.237.188:36823
I20260808 10:49:25.985037 3199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.237.188:36823 every 8 connection(s)
I20260808 10:49:25.985860 3095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/data/info.pb
I20260808 10:49:25.988168 2951 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } attempt: 3
I20260808 10:49:25.989260 3200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:49:25.993499 3200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:49:25.995007 32694 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu as pid 3095
I20260808 10:49:25.995155 32694 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDI8wVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786185862616645-32694-0/minicluster-data/master-2/wal/instance
I20260808 10:49:25.995860 32694 external_mini_cluster.cc:1428] Setting key 47c9127e00b0760b89e50607c1437d9a
Aug 08 10:49:25 dist-test-slave-twwt krb5kdc[2816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.190@KRBTEST.COM for kudu/127.31.237.188@KRBTEST.COM
I20260808 10:49:26.000420 3080 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } attempt: 3
Aug 08 10:49:25 dist-test-slave-twwt krb5kdc[2816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.188@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:49:26.002483 2958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.988723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:40000)
0808 10:49:25.988892 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.988912 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:25.989012 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:25.991553 (+ 2541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:25.991557 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:25.991772 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:25.991887 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:25.991895 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.993325 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.993329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:25.993885 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:25.993896 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.994878 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.994883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:25.995014 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:25.995684 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:25.995704 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:25.997280 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:25.999754 (+ 2474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:25.999758 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:25.999760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:26.000028 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:26.000478 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:26.000481 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:26.000482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:26.000853 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:26.001925 (+ 1072us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:26.001931 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:26.002044 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 10:49:26.002139 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 10:49:26.002283 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
Aug 08 10:49:26 dist-test-slave-twwt krb5kdc[2816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.188@KRBTEST.COM
I20260808 10:49:26.005376 3203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.989514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40000 (local address 127.31.237.188:36823)
0808 10:49:25.989979 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:25.989995 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:25.990032 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:49:25.990880 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:25.990900 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:25.991270 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:25.991623 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:25.992069 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.993178 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.994014 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.994722 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.995046 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:25.997469 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:25.997476 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:25.997490 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:25.997529 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:25.999632 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.000153 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.000159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.000166 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.000324 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.001212 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.001215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.001217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.001774 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:26.002712 (+ 938us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:26.002907 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 10:49:26.003489 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":152,"threads_started":1}
I20260808 10:49:26.006843 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.995555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.190:32959 (local address 127.0.0.1:37654)
0808 10:49:25.995998 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:25.996055 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:25.996228 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:25.996705 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:25.996713 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:25.997253 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:25.997447 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:25.997464 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.998497 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.998500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:25.999577 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:25.999593 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.000183 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.000187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:26.000299 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:26.000842 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:26.000863 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:26.002733 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:26.004794 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:26.004803 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:26.004810 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:26.005156 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:26.005496 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:26.005501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:26.005504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:26.005601 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:26.006070 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:26.006079 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:26.006228 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 10:49:26.006380 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 10:49:26.006589 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":147,"threads_started":1}
Aug 08 10:49:26 dist-test-slave-twwt krb5kdc[2816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.189@KRBTEST.COM for kudu/127.31.237.188@KRBTEST.COM
I20260808 10:49:26.008011 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.995569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37654 (local address 127.31.237.190:32959)
0808 10:49:25.995718 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:25.995723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:25.996131 (+ 408us) server_negotiation.cc:407] Connection header received
0808 10:49:25.996351 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:25.996354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:25.996406 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:25.996471 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:25.997601 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.998318 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.999716 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.000058 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.000183 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:26.002882 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:26.002886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:26.002889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:26.002916 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:26.004679 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.005271 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.005274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.005276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.005367 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.005701 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.005704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.005706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.005938 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:26.006069 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:26.007635 (+ 1566us) server_negotiation.cc:299] Negotiation successful
0808 10:49:26.007769 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:49:26.008735 3208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:25.996874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40008 (local address 127.31.237.188:36823)
0808 10:49:25.997238 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:25.997247 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:25.997261 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:49:25.997294 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:25.997297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:25.997363 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:25.997536 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:25.998403 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:25.999051 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:25.999984 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.000322 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.000423 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:26.003620 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:26.003623 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:26.003626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:26.003649 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:26.005436 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.006991 (+ 1555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.006994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.006995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.007100 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.007403 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.007406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.007409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.007608 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:26.007773 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:26.008463 (+ 690us) server_negotiation.cc:299] Negotiation successful
0808 10:49:26.008572 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":86,"threads_started":1}
I20260808 10:49:26.010509 2951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd: Bootstrap starting.
I20260808 10:49:26.010892 3200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:49:26.012941 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:26.000867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:40014)
0808 10:49:26.001147 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:26.001164 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:26.001240 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:26.002068 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:26.002072 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:26.002292 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:26.002403 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:26.002411 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.004517 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.004521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:26.005356 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:26.005367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.006230 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.006234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:26.006317 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:26.006798 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:26.006813 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:26.008288 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:26.010697 (+ 2409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:26.010702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:26.010704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:26.010991 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:26.011413 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:26.011416 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:26.011418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:26.011533 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:26.012424 (+ 891us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:26.012428 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:26.012547 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 10:49:26.012629 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:49:26.012778 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185}
I20260808 10:49:26.013621 2951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:26.013825 3210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:26.001387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40014 (local address 127.31.237.188:36823)
0808 10:49:26.001762 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.001772 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.001787 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:49:26.001828 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.001833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.001886 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.002811 (+ 925us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.003006 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.004364 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.005785 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.006107 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.006208 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:26.008467 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:26.008471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:26.008474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:26.008498 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:26.010475 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.011198 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.011201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.011204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.011298 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.011654 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.011658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.011758 (+ 100us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.012304 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:26.013391 (+ 1087us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:26.013476 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 10:49:26.013589 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":111,"threads_started":1}
I20260808 10:49:26.014622 2951 log.cc:824] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:26.016604 2951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd: No bootstrap required, opened a new log
I20260808 10:49:26.016786 3080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc: Bootstrap starting.
I20260808 10:49:26.019223 3080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc: Neither blocks nor log segments found. Creating new log.
Aug 08 10:49:26 dist-test-slave-twwt krb5kdc[2816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.188@KRBTEST.COM for kudu/127.31.237.189@KRBTEST.COM
I20260808 10:49:26.020179 3080 log.cc:824] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc: Log is configured to *not* fsync() on all Append() calls
Aug 08 10:49:26 dist-test-slave-twwt krb5kdc[2816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.190@KRBTEST.COM
I20260808 10:49:26.020697 2951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } }
I20260808 10:49:26.020975 2951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bf7a7bb007940c5ae5c0035e00086dd, State: Initialized, Role: FOLLOWER
I20260808 10:49:26.021407 2951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } }
I20260808 10:49:26.022742 3080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc: No bootstrap required, opened a new log
I20260808 10:49:26.023257 2951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:26.025537 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:26.010545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37658 (local address 127.31.237.190:32959)
0808 10:49:26.010686 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.010690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.010703 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:49:26.010770 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.010774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.010831 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.010881 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.011712 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.012462 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.013704 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.014032 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.014106 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:26.021246 (+ 7140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:26.021253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:26.021259 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:26.021290 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:26.023059 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.023646 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.023649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.023651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.023737 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.024076 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.024079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.024081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.024302 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:26.025166 (+ 864us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:26.025244 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 10:49:26.025370 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:49:26.025647 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:26.011390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.189:36265 (local address 127.0.0.1:43860)
0808 10:49:26.011759 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:26.011772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:26.011856 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:26.012098 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:26.012101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:26.012268 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:26.012355 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:26.012364 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.013371 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.013374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:26.014174 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:26.014184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.015077 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.015081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:26.015177 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:26.018750 (+ 3573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:26.018766 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:26.021058 (+ 2292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:26.023712 (+ 2654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:26.023715 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:26.023717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:26.023979 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:26.024344 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:26.024349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:26.024352 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:26.024451 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:26.025027 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:26.025030 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:26.025162 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 10:49:26.025285 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 10:49:26.025440 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301}
I20260808 10:49:26.026450 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:26.011630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43860 (local address 127.31.237.189:36265)
0808 10:49:26.011738 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.011742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.011817 (+ 75us) server_negotiation.cc:407] Connection header received
0808 10:49:26.011936 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.011939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.011985 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.012084 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.012475 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.013230 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.014306 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.014935 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.015067 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:26.021243 (+ 6176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:26.021247 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:26.021250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:26.021279 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:26.023546 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.024113 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.024119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.024121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.024220 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.024573 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.024576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.024578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.024854 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:26.024970 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:26.026161 (+ 1191us) server_negotiation.cc:299] Negotiation successful
0808 10:49:26.026312 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 10:49:26.027285 3200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:49:26.027200 3080 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } }
I20260808 10:49:26.027462 3080 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbadfbf3016e4d03ba0bea28347ad7fc, State: Initialized, Role: FOLLOWER
I20260808 10:49:26.028031 3080 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } }
I20260808 10:49:26.029516 3080 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:26.031096 3214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } } }
I20260808 10:49:26.031250 3214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 10:49:26 dist-test-slave-twwt krb5kdc[2816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.237.188@KRBTEST.COM for kudu/127.31.237.188@KRBTEST.COM
I20260808 10:49:26.029400 2837 master_runner.cc:186] Error getting master registration for 127.31.237.189:36265: OK, instance_id { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" instance_seqno: 1786186165741537 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:49:26.033860 2837 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:49:26.034409 3235 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } } }
Aug 08 10:49:26 dist-test-slave-twwt krb5kdc[2816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786186165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.237.189@KRBTEST.COM
I20260808 10:49:26.034555 3235 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:49:26.035876 3234 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:49:26.035954 3234 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:49:26.038841 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:26.027734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.237.188:36823 (local address 127.0.0.1:40022)
0808 10:49:26.028010 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 10:49:26.028024 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:49:26.028099 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:49:26.028523 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:49:26.028527 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:49:26.028706 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:49:26.028798 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:26.028805 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.029758 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.029761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:26.030535 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:49:26.030545 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.031252 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.031257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:49:26.031327 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:26.032075 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:49:26.032091 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:49:26.033642 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:49:26.037201 (+ 3559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:26.037204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:26.037206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:26.037455 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:26.037747 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:49:26.037750 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:49:26.037751 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:49:26.037833 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:49:26.038288 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:49:26.038293 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:49:26.038402 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 10:49:26.038506 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 10:49:26.038681 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260808 10:49:26.039443 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:26.027354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43864 (local address 127.31.237.189:36265)
0808 10:49:26.027743 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.027748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.027759 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:49:26.027793 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.027796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.027855 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.027926 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.028763 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.029471 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.030647 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.030996 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.031100 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:26.035407 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:26.035411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:26.035414 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:26.035443 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:26.037083 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.037599 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.037602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.037604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.037693 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.038023 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.038026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.038028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.038249 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:26.039075 (+ 826us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:26.039153 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 10:49:26.039284 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":308}
I20260808 10:49:26.039476 3210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:26.027800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40022 (local address 127.31.237.188:36823)
0808 10:49:26.028247 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.028251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.028262 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:49:26.028291 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.028293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.028336 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.028410 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.028915 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.029645 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.030693 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.031042 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.031136 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:26.033799 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:26.033802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:26.033805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:26.033829 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:26.037087 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.037562 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.037565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.037567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.037656 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.037919 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.037921 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.037923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.038156 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:26.038249 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:26.039219 (+ 970us) server_negotiation.cc:299] Negotiation successful
0808 10:49:26.039342 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385}
I20260808 10:49:26.040043 2965 master_runner.cc:186] Error getting master registration for 127.31.237.188:36823: OK, instance_id { permanent_uuid: "a06d85c752284403aac72e0391a3817e" instance_seqno: 1786186165982497 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:49:26.042371 2965 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:49:26.042794 3200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e: Bootstrap starting.
W20260808 10:49:26.044044 3250 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:49:26.044144 3250 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:49:26.045063 32694 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:49:26.045790 3200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e: Neither blocks nor log segments found. Creating new log.
I20260808 10:49:26.046288 2907 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2bf7a7bb007940c5ae5c0035e00086dd"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37658
I20260808 10:49:26.046552 2907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 10:49:26.046713 2907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:26.046965 3200 log.cc:824] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:49:26.048179 2907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } }
I20260808 10:49:26.049401 3200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e: No bootstrap required, opened a new log
I20260808 10:49:26.049820 3041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc"
I20260808 10:49:26.050076 3041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:26.052021 3041 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bf7a7bb007940c5ae5c0035e00086dd in term 1.
I20260808 10:49:26.052740 2867 leader_election.cc:302] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bf7a7bb007940c5ae5c0035e00086dd, bbadfbf3016e4d03ba0bea28347ad7fc; no voters:
I20260808 10:49:26.053109 3214 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:49:26.053360 3214 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [term 1 LEADER]: Becoming Leader. State: Replica: 2bf7a7bb007940c5ae5c0035e00086dd, State: Running, Role: LEADER
I20260808 10:49:26.053757 3214 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } }
I20260808 10:49:26.055045 3200 raft_consensus.cc:348] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } }
I20260808 10:49:26.055354 3200 raft_consensus.cc:729] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06d85c752284403aac72e0391a3817e, State: Initialized, Role: FOLLOWER
I20260808 10:49:26.055737 3200 consensus_queue.cc:260] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } }
I20260808 10:49:26.055826 3214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bf7a7bb007940c5ae5c0035e00086dd. Latest consensus state: current_term: 1 leader_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } } }
I20260808 10:49:26.055953 3214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:26.056406 2907 leader_election.cc:288] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [CANDIDATE]: Term 1 election: Requested vote from peers bbadfbf3016e4d03ba0bea28347ad7fc (127.31.237.189:36265), a06d85c752284403aac72e0391a3817e (127.31.237.188:36823)
I20260808 10:49:26.057308 3200 sys_catalog.cc:564] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:49:26.057346 3164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a06d85c752284403aac72e0391a3817e"
I20260808 10:49:26.057595 3164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:49:26.059058 3164 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bf7a7bb007940c5ae5c0035e00086dd in term 1.
I20260808 10:49:26.061309 3263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:49:26.062292 3263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:49:26.066812 3262 sys_catalog.cc:455] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } } }
I20260808 10:49:26.066941 3262 sys_catalog.cc:458] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:49:26.068619 3280 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:49:26.068702 3280 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:49:26.078890 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:26.066641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37666 (local address 127.31.237.190:32959)
0808 10:49:26.066774 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.066780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.069020 (+ 2240us) server_negotiation.cc:407] Connection header received
0808 10:49:26.069136 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.069139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.069200 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.069280 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.070308 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.071161 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.072375 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.072708 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.072817 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:26.074800 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:26.074814 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:26.074817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:26.074847 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:26.076642 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.077225 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.077228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.077230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.077324 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.077651 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.077654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.077656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.077906 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:26.078046 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:26.078457 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 10:49:26.078598 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 10:49:26.088531 3210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:26.067655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40036 (local address 127.31.237.188:36823)
0808 10:49:26.067770 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.067776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.075260 (+ 7484us) server_negotiation.cc:407] Connection header received
0808 10:49:26.076432 (+ 1172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.076436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.076500 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.076553 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.078714 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.079489 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.081999 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.082354 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.082458 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:26.084482 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:26.084486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:26.084489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:26.084520 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:26.086476 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.087036 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.087040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.087042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.087134 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.087421 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.087424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.087426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.087633 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:26.087760 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:26.088187 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 10:49:26.088357 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:49:26.091133 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:26.067279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43868 (local address 127.31.237.189:36265)
0808 10:49:26.067402 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.067406 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.074913 (+ 7507us) server_negotiation.cc:407] Connection header received
0808 10:49:26.075352 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.075356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.075413 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.075478 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.077408 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.078224 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.081144 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:26.081515 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:26.081621 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:26.083791 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:49:26.083795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:49:26.083798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:49:26.083828 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:49:26.085579 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.086086 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.086090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.086092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.086176 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:49:26.086475 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:49:26.086479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:49:26.086480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:49:26.086721 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:49:26.086837 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:26.090823 (+ 3986us) server_negotiation.cc:299] Negotiation successful
0808 10:49:26.090959 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:49:26.092331 3164 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e [term 1 FOLLOWER]: Refusing update from remote peer 2bf7a7bb007940c5ae5c0035e00086dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:49:26.093109 3259 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:49:26.098628 3262 sys_catalog.cc:455] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bf7a7bb007940c5ae5c0035e00086dd. Latest consensus state: current_term: 1 leader_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } } }
I20260808 10:49:26.098771 3262 sys_catalog.cc:458] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:49:26.098883 3041 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc [term 1 FOLLOWER]: Refusing update from remote peer 2bf7a7bb007940c5ae5c0035e00086dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:49:26.099499 3259 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:49:26.103371 3235 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bf7a7bb007940c5ae5c0035e00086dd. Latest consensus state: current_term: 1 leader_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } } }
I20260808 10:49:26.103506 3235 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:49:26.114533 3259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } } }
I20260808 10:49:26.114712 3259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:26.116144 3294 mvcc.cc:205] Tried to move back new op lower bound from 7316218536301002752 to 7316218536160440320. Current Snapshot: MvccSnapshot[applied={T|T < 7316218536301002752}]
I20260808 10:49:26.116293 3262 sys_catalog.cc:455] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } } }
I20260808 10:49:26.116420 3262 sys_catalog.cc:458] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:49:26.117221 3214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } } }
I20260808 10:49:26.117345 3214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd [sys.catalog]: This master's current role is: LEADER
I20260808 10:49:26.118489 3235 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bf7a7bb007940c5ae5c0035e00086dd" member_type: VOTER last_known_addr { host: "127.31.237.190" port: 32959 } } peers { permanent_uuid: "bbadfbf3016e4d03ba0bea28347ad7fc" member_type: VOTER last_known_addr { host: "127.31.237.189" port: 36265 } } peers { permanent_uuid: "a06d85c752284403aac72e0391a3817e" member_type: VOTER last_known_addr { host: "127.31.237.188" port: 36823 } } }
I20260808 10:49:26.118630 3235 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:49:26.119124 3301 mvcc.cc:205] Tried to move back new op lower bound from 7316218536301002752 to 7316218536160440320. Current Snapshot: MvccSnapshot[applied={T|T < 7316218536301002752}]
I20260808 10:49:26.116326 3295 mvcc.cc:205] Tried to move back new op lower bound from 7316218536301002752 to 7316218536160440320. Current Snapshot: MvccSnapshot[applied={T|T < 7316218536301002752}]
I20260808 10:49:26.127746 3263 catalog_manager.cc:1347] Generated new cluster ID: 834b2ddca98d4bf8802ab8e1e11f530f
I20260808 10:49:26.127836 3263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:49:26.153069 3263 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:49:26.154824 3263 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:49:26.179154 3263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bf7a7bb007940c5ae5c0035e00086dd: Generated new TSK 0
I20260808 10:49:26.180220 3263 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:49:26.213920 2957 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:26.210153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37674 (local address 127.31.237.190:32959)
0808 10:49:26.210292 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.210297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.212347 (+ 2050us) server_negotiation.cc:407] Connection header received
0808 10:49:26.212929 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.212934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.212997 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.213056 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.213776 (+ 720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37674: BlockingRecv error: recv got EOF from 127.0.0.1:37674 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 10:49:26.215338 3091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:26.210990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43874 (local address 127.31.237.189:36265)
0808 10:49:26.211119 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.211124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.212758 (+ 1634us) server_negotiation.cc:407] Connection header received
0808 10:49:26.214506 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.214509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.214565 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.214641 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.215216 (+ 575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43874: BlockingRecv error: recv got EOF from 127.0.0.1:43874 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 10:49:26.217500 3210 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:26.211687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40038 (local address 127.31.237.188:36823)
0808 10:49:26.211814 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.211818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.215930 (+ 4112us) server_negotiation.cc:407] Connection header received
0808 10:49:26.216559 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.216563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.216623 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.216683 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.217314 (+ 631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40038: BlockingRecv error: recv got EOF from 127.0.0.1:40038 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 10:49:26.222833 3091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:26.221014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43886 (local address 127.31.237.189:36265)
0808 10:49:26.221175 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.221179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.221747 (+ 568us) server_negotiation.cc:407] Connection header received
0808 10:49:26.221986 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.221990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.222075 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.222133 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.222713 (+ 580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43886: BlockingRecv error: recv got EOF from 127.0.0.1:43886 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260808 10:49:26.228132 3091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:26.226487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43892 (local address 127.31.237.189:36265)
0808 10:49:26.226591 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.226596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.227176 (+ 580us) server_negotiation.cc:407] Connection header received
0808 10:49:26.227385 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.227388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.227443 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.227498 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.228033 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43892: BlockingRecv error: recv got EOF from 127.0.0.1:43892 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 10:49:26.234778 3091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:26.233192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43896 (local address 127.31.237.189:36265)
0808 10:49:26.233300 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.233304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.233836 (+ 532us) server_negotiation.cc:407] Connection header received
0808 10:49:26.234022 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.234025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.234079 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.234130 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.234682 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43896: BlockingRecv error: recv got EOF from 127.0.0.1:43896 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 10:49:26.242372 3091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:26.240833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43908 (local address 127.31.237.189:36265)
0808 10:49:26.240956 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.240960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.241473 (+ 513us) server_negotiation.cc:407] Connection header received
0808 10:49:26.241649 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.241653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.241711 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.241760 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.242253 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43908: BlockingRecv error: recv got EOF from 127.0.0.1:43908 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 10:49:26.254810 3091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:26.253082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.31.237.189:36265)
0808 10:49:26.253195 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.253200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.253816 (+ 616us) server_negotiation.cc:407] Connection header received
0808 10:49:26.253935 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.253939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.254012 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.254129 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.254685 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43918: BlockingRecv error: recv got EOF from 127.0.0.1:43918 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 10:49:26.273849 3091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:26.272668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43926 (local address 127.31.237.189:36265)
0808 10:49:26.272827 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.272832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.272980 (+ 148us) server_negotiation.cc:407] Connection header received
0808 10:49:26.273101 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.273105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.273169 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.273262 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.273722 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43926: BlockingRecv error: recv got EOF from 127.0.0.1:43926 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 10:49:26.309280 3091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:26.308012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43942 (local address 127.31.237.189:36265)
0808 10:49:26.308179 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.308184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.308329 (+ 145us) server_negotiation.cc:407] Connection header received
0808 10:49:26.308523 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.308527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.308587 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.308679 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.309157 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43942: BlockingRecv error: recv got EOF from 127.0.0.1:43942 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 10:49:26.376772 3091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:26.375582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43950 (local address 127.31.237.189:36265)
0808 10:49:26.375767 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.375771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.375790 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:49:26.375949 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.375952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.376003 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.376079 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.376640 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43950: BlockingRecv error: recv got EOF from 127.0.0.1:43950 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260808 10:49:26.508387 3091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:26.507072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43952 (local address 127.31.237.189:36265)
0808 10:49:26.507234 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.507239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.507432 (+ 193us) server_negotiation.cc:407] Connection header received
0808 10:49:26.507624 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.507627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.507676 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.507770 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.508264 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43952: BlockingRecv error: recv got EOF from 127.0.0.1:43952 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 10:49:26.768105 3091 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:49:26.766841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43956 (local address 127.31.237.189:36265)
0808 10:49:26.767004 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:26.767010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:26.767114 (+ 104us) server_negotiation.cc:407] Connection header received
0808 10:49:26.767309 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:26.767312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:26.767374 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:26.767470 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:49:26.767982 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43956: BlockingRecv error: recv got EOF from 127.0.0.1:43956 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 10:49:27.045570 3250 catalog_manager.cc:1259] Loaded cluster ID: 834b2ddca98d4bf8802ab8e1e11f530f
I20260808 10:49:27.045715 3250 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc: loading cluster ID for follower catalog manager: success
I20260808 10:49:27.047432 3250 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc: acquiring CA information for follower catalog manager: success
I20260808 10:49:27.048645 3250 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bbadfbf3016e4d03ba0bea28347ad7fc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:49:27.070111 3280 catalog_manager.cc:1259] Loaded cluster ID: 834b2ddca98d4bf8802ab8e1e11f530f
I20260808 10:49:27.070194 3280 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e: loading cluster ID for follower catalog manager: success
I20260808 10:49:27.071938 3280 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e: acquiring CA information for follower catalog manager: success
I20260808 10:49:27.073110 3280 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a06d85c752284403aac72e0391a3817e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:49:27.286137 3421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:27.282320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43970 (local address 127.31.237.189:36265)
0808 10:49:27.282773 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:27.282782 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:27.282798 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:49:27.282854 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:27.282857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:27.283003 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:27.283107 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:49:27.283358 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:27.283872 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:27.284684 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:27.284897 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:27.284994 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:27.285257 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:49:27.285706 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:49:27.285802 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:27.285858 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 10:49:27.285907 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":104,"threads_started":1}
I20260808 10:49:27.293167 3429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:27.289328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40052 (local address 127.31.237.188:36823)
0808 10:49:27.289621 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:27.289628 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:27.289644 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:49:27.289738 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:27.289740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:27.289856 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:27.289990 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:49:27.290216 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:27.290827 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:27.291651 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:27.291890 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:27.292001 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:27.292310 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:49:27.292739 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:49:27.292879 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:27.292942 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 10:49:27.292999 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":85,"threads_started":1}
I20260808 10:49:27.300971 3421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:27.296932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43984 (local address 127.31.237.189:36265)
0808 10:49:27.297076 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:27.297083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:27.297157 (+ 74us) server_negotiation.cc:407] Connection header received
0808 10:49:27.297350 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:27.297353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:27.297417 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:27.297471 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:49:27.297806 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:27.298491 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:27.299730 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:27.300026 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:27.300138 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:27.300452 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:49:27.300534 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:49:27.300618 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:27.300740 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 10:49:27.300801 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:49:27.301302 3429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:27.297301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40064 (local address 127.31.237.188:36823)
0808 10:49:27.297438 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:27.297445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:27.297717 (+ 272us) server_negotiation.cc:407] Connection header received
0808 10:49:27.297898 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:27.297901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:27.297952 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:27.298025 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:49:27.298296 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:27.298971 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:27.299896 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:27.300230 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:27.300320 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:27.300701 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:49:27.300776 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:49:27.300841 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:27.301106 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 10:49:27.301172 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:49:27.301401 3439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:49:27.296499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37684 (local address 127.31.237.190:32959)
0808 10:49:27.296769 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 10:49:27.296776 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:49:27.296793 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:49:27.297021 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:49:27.297026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:49:27.297201 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:49:27.297742 (+ 541us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:49:27.297864 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:27.298677 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:27.299780 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:49:27.300137 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:49:27.300291 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:49:27.300617 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:49:27.300974 (+ 357us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:49:27.301080 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:49:27.301166 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 10:49:27.301222 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":86,"threads_started":1}
I20260808 10:49:27.304441 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 2837
I20260808 10:49:27.323943 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 2965
I20260808 10:49:27.338291 32694 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDI8wVd/build/debug/bin/kudu with pid 3095
2026-08-08T10:49:27Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3410 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6859 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (304715 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST